#tools-and-devops

1 messages · Page 65 of 1

urban pecan
#

okay, i'm down to this:

#

RuntimeWarning: Please specify version and hash for auto-installation of 'example-pypi-package'.
To get some valuable insight on the health of this package, please check out https://snyk.io/advisor/python/example-pypi-package
If you want to auto-install the latest version: use("example-pypi-package", version="0.1.0", hash_value="ce89b1fe92abc55b4349bc58462ba255c42132598df6fe3a416a75b39b872a77", auto_install=True)

Snyk Advisor
View example-pypi-package on Snyk Open Source Advisor

Find the best open-source package for your project with Snyk Open Source Advisor. Explore over 1 million open source packages.

#

any suggestions on how to make this more useful?

urban pecan
#

so, when this works as it's supposed to, you can just distribute a minimal bootstrap and the rest would be pulled from pypi or some online repo

#

joy 🙂

#

no more dependency hell \o/

urban pecan
#

i wonder if it might actually be faster on a modern machine to keep the wheel zipped up and decompress every startup rather than reading the whole unzipped package from disk

urban pecan
#

test = use("example-pypi-package.examplepy", version="0.1.0", hash_value="ce89b1fe92abc55b4349bc58462ba255c42132598df6fe3a416a75b39b872a77", auto_install=True)
assert str(test.Number(2)) == "2"

#

works 🙂

#

sadly, numpy is being a diva about it, not sure why

urban pecan
#

probably because zipimport refuses to load .pyd and .so?

urban pecan
#

unzipping in a subfolder and using that fixes some issues, but numpy still doesn't like it

#

strange.. oh well

#

realising inline auto-install in general is a job well done for a day

urban pecan
#

released a new version of https://pypi.org/project/justuse/ - auto-install should work now well enough for pure python packages, c-extensions will have to wait a bit longer

PyPI
justuse

A self-documenting, functional way to import modules in Python with advanced features.

floral smelt
#

I've posted a docker issue with my django container days ago

#

And apparently it's bc Docker shell can't recognize the format

#

But when I try my scripts with bash it doesn't work

gleaming forge
#

How do I specify to ignore a directory and everything in it in my gitignore?

#

I thought just dir/ worked, but doesn't seem to be doing it

#

Hmm, ok nevermind

#

Seems it was just VSC being weird

burnt ibex
#

Sometimes you need to clear your git cache if that directory has been committed before

proven sapphire
#

Hey everyone, how can I get the number of physical hard disks from a python file? I thought maybe I could use psutil but there doesn't seem to be anything for it--I'm looking for an agnostic solution...right now I'm using psutil.disk_partitions() but I don't want the number of partitions, just the actual disks. I'm not entirely sure how disk_partitions() works, but I don't want the number of partitions, just the number of installed disks, so is that just the amount of devices in the disk_partitions array?

flat path
#

Quick question about nox, do you have to run it with the current working directory being the one containing the noxfile.py? It's not that annoying, but it's I something I noticed compared to tox. (please ping me since my attention is somewhere else)

waxen axle
livid valley
#

Can someone teach me the best (industry standard) way of using Pyenv and managing Pip packages?

# 
~ ❯ which python                                                               
/Users/Mushu/.pyenv/shims/python
~ ❯ which pip                                                                   
/usr/local/bin/pip
#

I have multiple versions of Python installed using Pyenv. I need some older Python versions for a class I'm taking. The issue is that my global python cannot see any pip packages... which makes sense because numpy for Python 3.6 should be different that numpy for Python 3.9

#

How the heck do people manage pip packages? Is there a way to point pip to the global pyenv folder?

tawny temple
#

Doesn't pyenv make pip point to the currently active Python version?

#

You shouldn't try to share packages across Python versions, since not all packages are compatible with different versions.

#

If you need a package in multiple versions, just install it for each version

livid valley
#

Yep, I understand that. But how do developers actually manage pip packages?
Clearly, it doesn't make sense to duplicate your entire library of pip packages if you move from 3.9.4 to 3.9.5

#

I've been frozen for 2 days, and reddit tells me that Poetry is the new, cool kid for package management.

dusty maple
#

For upgrading pyenv versions I usually just pip freeze then reinstall on the newer Python version, also it's quite odd that pip is not pointing to the shims directory, does it exist there?

tawny temple
#

If you install your packages to the user site, they should remain accessible to all Python versions with the same minor version (i.e. patch versions won't require reinstalling packages)

#

Since the user site is categorised by minor Python version

dusty maple
#

TIL actually, thanks

tawny temple
#

No problem

wooden ibex
tawny temple
#

To be clear, a user site means installing packages with pip install --user

livid valley
#

I think my config (.zshrc file) is messed up because, yes, I think pip should be pointing to the shims directory.
Maybe the solution is to clean reinstall my entire Mac and wipe off any bad pointers.

wooden ibex
#

For production, docker is way to go for sure

tawny temple
#

Yeah I understood this as a local development question. It wouldn't make sense to be using something like pyenv in prod.

wooden ibex
#

For local development, what IDE do you use Mushu?

#

If VSCode, docker as well for development

#

If PyCharm, fight with them for that feature

livid valley
#

I use VS Code... I've never done any real production before. I need Python 3.6 for a testsuite in my university class. However, most of my hobby development is done in 3.9

wooden ibex
#

Learn it, fall in love, refuse to convert to PyCharm because they don’t have that feature

tawny temple
#

What is the advantage over using pyenv

wooden ibex
wooden ibex
#

It’s 100% isolated environment

#

So one project can’t mess with another

livid valley
#

So, I'm leaning towards a clean install of my Mac, then use Docker to avoid dependency hell issues.

What is the general method for installing Python into a Docker container? Should my Mac have python3?

tawny temple
#

I've never experienced my project environment conflicting with each other. Maybe Docker makes sense for some projects that require more complex provisioning, but most of the time a simple venv will do. Then it just becomes a matter of installing docker and configuring a dockerfile, or installing pyenv and creating a venv. Or is there more to it that I am not seeing?

#

If you want to use Docker, then all you need is Docker. Doesn't matter if the host system has Python or not. You'll be using a base Docker image to get the Python version you need.

livid valley
#

Ah I see.

wooden ibex
#

it's 100% isolated environment, nothing can fight, not python versions, not pip versions nothing

#

someone accidently removed Python version, your fine, upgrading python, change out JSON file and rebuild dev container

#

did someone forget to activate virtualenv, it doesn't matter

tawny temple
#

I suppose those are all valid points technically.

#

Maybe they are issues for people

#

But I have personally never encountered any of that

wooden ibex
#

also, it takes like an hour to learn

#

you never needed two version of python for old code and new?

tawny temple
#

You mean in the same project?

wooden ibex
tawny temple
#

Yeah. I use pyenv and each project has its own venv

#

I mean that I never encounter things "fighting" me or Python versions getting removed, etc.

wooden ibex
#

Sure, but if you decided to remove Python because you ran into a problem, this comes up

tawny temple
#

I don't see why I would remove Python

wooden ibex
#

my point about Dev Containers is they are so easy to use, once you adopt them, it's awesome

tawny temple
#

While there may be more avenues for fucking up with venvs/pyenv, it doesn't seem difficult to avoid that (I am a living example of that).

shadow furnace
#

@proven sapphire

unknown.png
shadow furnace
#

uh should work on ubuntu based systems and other unix distros python comes preinstalled anyway, try it out open your terminal and type this command python3 -c "import socket; print(socket.getfqdn())"

#

@proven sapphire

latent narwhal
#

how do i stop prettier from going to the next line, for eg this is before saving and this is after saving

unknown.png
thick bear
#

Guys, wasn't sure where to ask this. I am running windows, and I want to create a good development environment, that I could make an image of so it would be easier to bootstrap it anytime I need a new environment. Although I do fancy the idea of creating it myself, are there already scripts/bootstrapped vm's that I could just download? Or, would you recommend a certain OS/flavour for creating a good dev environment in VMWare or virtualbox? I am looking to code mostly in Python, and my linux cli skills are (I hope) on point

rapid sparrow
#

all installation to organize my working place can be actually automatized to be made in a matter of minutes with one command only

#

but I did not reach that level of laziness, automated only some parts

lethal flax
#

I would like to use Sphinx to generate rich email content (from release notes which are in RST format, among other info).
Is there an existing Sphinx template which generates everything into a single file? No external CSS being loaded, just all embedded in one html doc?

rapid sparrow
#

i did not try that, but I guess it is a worthy to try.

#

huh, I have actually Sphinx ready to try that

#

welllll....

#

it surely made a single file... but css and js remained being separately

#

you would have to add them on your own I guess

#

singlehtml builder just combined content from multiple RST files into one html

#

probably somewhere else may be configuration exists to include css/js into resulting html

tawdry needle
# wooden ibex my point about Dev Containers is they are so easy to use, once you adopt them, i...

i use containers at work now with docker compose, and it's definitely a mixed bag. volumes are fussy (no access to volume during container build). debugging requires different tooling. i need a "shadow env" on my local system to get IDEs to work right (although i'm working on getting Pycharm professional edition, which has good Python-in-Docker support). docker is sometimes still slow doing this or that operation, even with build layer caching.

#

i don't hate the workflow, but it's not an equivalent dev experience, at least not for me. and definitely more complicated than 1 hour of self study

#

pyenv-virtualenv takes care of most of what i need

rapid sparrow
#

perhaps this could help

floral wing
#

is creating automation considered devops?

rapid sparrow
#

provide an example

#

I would say, devops creates automations, but not ever automation is devops

floral wing
#

automation like playbook run jobs to either create metrics, make changes/additions/removals to the infrastructure and other report related tasks

rapid sparrow
#

you are changing infrastructure as a code, which is basically devops is

floral wing
#

thanks! in that case, is it a bad place for an absolute beginner to start? it may or may not be required of me soon is why i ask

urban pecan
#

if you may need it, it's a good a corner to start as any

shy locust
#

yup

#

Its nothing to be afraid of

kindred vapor
tidal iron
#

Hi I have a precommit hook for yapf. After adding a pyproject.toml file I am getting an error-
toml package is required
If I remove pyproject.toml then this error goes away. Why is this happening and is there any fix?

burnt ibex
#

Have you run poetry install?

tidal iron
#

I'm using flit

burnt ibex
#

It seems like it's looking for the toml package in on pypi and not finding it

burnt ibex
tidal iron
#

Yes

burnt ibex
#

I've never used flit, so I dunno

final wing
#

pipenv or poetry? Which do you recommend?

burnt ibex
#

I prefer poetry as it's much faster in my experience

final wing
#

Faster? At what?

burnt ibex
#

Installing dependencies occurs faster with poetry

#

I've had it take a very long time with pipenv

final wing
#

That's a bit puzzling. How fast dependencies are installed should depend mainly on your network connection?

burnt ibex
#

I dunno. Poetry just does its faster for me, and a lot of other people

#

It may have to do with how it caches and things like that

final wing
#

Okay, thanks for your input.

#

To be fair, I had some slight experience with conda, and it wasn't fun. It can take ages, and seems to break easily.

#

OTOH those envs were pretty huge.

tawdry needle
#

conda is known to be slow and somewhat fragile

urban pecan
#

@tawdry needle on the note of deployment and containers, what would you say is the main pain point of current solutions? speed of downloads? what could sway you to use() auto-install a package?

tawdry needle
#

personally? nothing

#

(sorry)

urban pecan
#

aww.

tawdry needle
#

the main point of containers is isolation

#

i know exactly what programs are installed, and what versions

#

i know exactly what user accounts are present

#

and i can deploy that exact configuration anywhere, maybe even in a cluster (like kubernetes)

#

isolation, control, reproducibility, easy to deploy

urban pecan
#

well, but you could run different versions of a package in the same program

#

and a shared package would still be hash-pinned, so there's no overlap

#

so you don't need to download the same package over and over?

#

(argument for .so libs, i guess)

tawdry needle
#

You have to distinguish between "distribution package" and "python package" here

#

You can have 10 different versions of the former, but only one version of the latter at a time with the same name

#

Unless this is what your custom import machinery does

#

So you could have foo_1_0_2 = use('foo', version='1.0.2), but even then you'd have to do some pretty serious black magic to make sure name resolution works inside thr foo package

urban pecan
#

the nomenclature is problematic, to say the least. a "distribution" is a defined file ("wheel") on pypi containing a package with a certain hash

#

if you query pypi for a package name with a certain version, it gives you a list of all files with their hashes

#

that match that name and version

#

also the corresponding url

#

foo_1_0_2 = use('foo', version='1.0.2) would only use the system/pip-installed stuff and check if the version matches, anything beyond that requires the auto_install=True arg

#

so for instance

#

test = use("example-pypi-package.examplepy", version="0.1.0", hash_value="ce89b1fe92abc55b4349bc58462ba255c42132598df6fe3a416a75b39b872a77", auto_install=True)

#

would get you exactly that file

#

and import examplepy from that package

vital sentinel
#

[crossposted from #help-chestnut ]
how do you easily share code across two projects?

In Rust for example I would extract the common code into a separate directory and then add a dependency entry to all dependant projects that points to ../common_code

The equivalent in Python seems to involve modifying PYTHONPATH which seems hacky and makes me think this is not the Pythonic way to share code across projects locally. What is the canonical way to do it?

urban pecan
#

well, i don't know about "canonical" at this point, but this would be an excellent case for justuse 🙂

vital sentinel
#

what is justuse?

urban pecan
vital sentinel
#

i cant find anything relevant

Screenshot_20210701_183115.png
#

ah thanks

urban pecan
#

it's the thing i've been working on and discussing with @tawdry needle 😉

vital sentinel
#

oh so it's like an in development thing

urban pecan
#

yeah

#

in this case, you can use(use.Path("some/path/to/foo.py"))

#

or use an url to github

#

if you really want some canonical solution, you'd probably go for an isolated environment, pip install stuff

#

messing with pythonpath is really not a good idea

vital sentinel
#

I might just

#

symlink the files into the second project

tawdry needle
tawdry needle
#

symlinking is uglier than setting pythonpath imo

#

e.g. my company's monorepo is kind of like this.

when you're working on service1, you have a venv for that specific service, and you do pip install -e ../shared/python && pip install -e . to get everything into the venv.

monorepo/
  common/
    python/
      setup.py
      src/my_org/
        __init__.py
        utils.py

  service1/
    src/
      service1/
        __init__.py
        __main__.py
        app.py

  service2/
    src/
      service2/
        app.py
        __init__.py
        __main__.py
urban pecan
tawdry needle
vital sentinel
#

sorry I had to go for a minute

#

my project is pretty simple, these are the files ```
main.py
dsbsession_web.py
util.py
dsbsession_android.py

urban pecan
# tawdry needle i am using "python package" specifically to mean "a module that contains other m...

that should be covered by the example i posted above - https://pypi.org/project/example-pypi-package/ ( https://github.com/tomchen/example_pypi_package )

PyPI
example-pypi-package

Example PyPI (Python Package Index) Package

GitHub
tomchen/example_pypi_package

Example PyPI (Python Package Index) package set up with automated tests and publishing using GitHub Actions CI/CD, primarily for GitHub + VS Code (Windows / Mac / Linux) users - tomchen/example_pyp...

vital sentinel
#

lemme read through what you two wrote ^^

#

those seem to be good solution to large scale projects

tawdry needle
vital sentinel
#

I think I'll end up either symlinking, modifying PYTHONPATH or doing the sys.path.insert() hack

#

just to get this roadblock over with

tawdry needle
#

i'd prefer PYTHONPATH over the other 2 options

urban pecan
# tawdry needle i'm not sure what you mean by this

test = use("example-pypi-package.examplepy", version="0.1.0", hash_value="ce89b1fe92abc55b4349bc58462ba255c42132598df6fe3a416a75b39b872a77", auto_install=True) <- this is a pure python package with "examplepy" as subpackage and __init__ etc

vital sentinel
tawdry needle
#

@urban pecan now you're being unclear 😉 . example-pypi-package is a "distribution" and examplepy is a "package".

urban pecan
#

the use() call basically corresponds to "pip install example-pypi-package hash_mode..." and "from example-pypi-package import examplepy"

tawdry needle
#

there is no example-pypi-package to import from

urban pecan
#

ahhh 🙂

tawdry needle
#

example-pypi-package exists only to pip and setuptools, python itself has no knowledge of it

urban pecan
#

right!

tawdry needle
#

unless you dump all packages in the distribution into a top level package

#

which would totally break namespace packages and a few other things (i think?)

urban pecan
#

no, it wouldn't because the import machinery does that name mangling either way, i'm just mirroring that behaviour for strings (which sucks!)

tawdry needle
#

what name mangling?

#

if you do pip install example-pypi-package, it installs examplepy into site-packages

urban pecan
#

an import examplepy is ambiguous

#

because it may either mean an actual "examplepy.py" module or a package

tawdry needle
#

python has clear semantics for handling this

#

it looks for packages first, and then modules, i think

#

this is documented somewhere, i've read it before

#

ultimately "installing" a python package amounts to copying it into the correct site-packages directory

urban pecan
#

well.. yeah, it tries to do the right thing, but it's ambiguous because you can shadow things in sys.path

#

also you can't force it to import a package if you have the same named module etc

#

sorry, that's just me ranting

tawdry needle
#

so let's say you do some clever name mangling and now you have isolated site-packages, neither of which is in sys.path by default

/path/to/justuse/data
  site-packages_example-pypi-package_v0.1.0/
    examplepy/
      __init__.py
  site-packages_example-pypi-package_v0.2.3/
    examplepy/
      __init__.py
      utils.py
urban pecan
#

point is, use() refuses to guess. for use() with str it's always a package

tawdry needle
#

you still have to un-break name resolution inside both examplepys

urban pecan
#

and if you want to import a module, you'd use it with a pathlib Path

tawdry needle
#

fine, but that doesn't solve the problem i'm talking about here

#

which has to do with name resolution

#

when python execs the contents of examplepy/__init__.py, it might contain from examplepy.utils import stuff

#

you need to make sure python knows where to find examplepy, and that it's the right examplepy, from the right version of the right distribution

#

i don't know if your code already does that. but that's what i was trying to explain before

#

maybe it's as simple as tacking on the temporary/isolated site-packages directory to the search path while you're loading the specific package

#

this could actually be one way to isolate dependencies, npm-style

#

there could be other issues there, e.g. it'd totally break pickling

urban pecan
#

well, the problem is that import only knows about the "official" stuff and pip refuses to install multiple versions, so any time you import stuff, it only can find the official package. so, one solution i figured out (but not implemented yet) would be to pass in an "import_to_use" argument

#

which would catch import statements and translate into use() calls if they match a certain name

#

that way you could resolve sub-dependencies to specific pinned versions

#

and it would still be compatible with the classic pip way

#

as i said in #databases, one big unresolved question is how to have both, hash-pinned versioning and still be able to quickly apply security patches, mostly automatic

#

the latter basically is the complete opposite of hash-pinning

tawdry needle
#

the way i was proposing, you just don't use import for these

#

but again you've spent a lot more time inside the import machinery than i have

#

so maybe "just" setting sys.path while exec'ing the module isn't enough

urban pecan
#

actually, switching CWDs is a much better answer

#

works well with import and use, no sys.path hacking necessary

#

the way i solved it in use() is to recursively switch the CWD and switch back to the original after import

urban pecan
#

quite honestly, sys.path was a big mistake to begin with. if it had been a dict with some metadata...

#

even zipimport is limited to pure python packages because of that, afaik

#

we had some serious problems because of that limitation because numpy uses .so files that zipimport doesn't want to handle, now we need to manually unzip and link those files to be able to handle c-extensions.. it could've been so nice and simple :/

#

it's all connected and boils down to those fundamental limitations of the import system

tawdry needle
#

ah right, you did tell me about the CWD thing

urban pecan
#

sorry for the rant 🙂

#

it was really quite frustrating to solve those issues correctly without resorting to sys.path hacking, but i think it was worth the trouble

tawdry needle
#

I guess I don't quite understand what a hack about it, from a high level it makes sense to have a "search path"

#

I know you said some of the internal machinery was hacky

#

I haven't spent too much time trying to understand it, I know there is some special "loader" machinery

#

Sounds kind of similar to how it works in Lua

urban pecan
#

for one, trying to load everything in the sys.path to see if it matches is just ugly from a proper design perspective. secondly, anything can change the sys.path anywhere, so it's down to gambling that your code finds the right thing

#

for example, i ran into troubles once because i appended my stuff to sys.path

#

it worked when i prepended it

#

now i test my use() code by del sys.path and see if things work without it at all 🙂

#

which it does!

urban pecan
urban elbow
#

I'm struggling to get my github workflow to upload to pypi working, it's a mixed rust and python project.
action i'm using: https://paste.pythondiscord.com/lehelevuxe.yml?noredirect
The action being run: https://github.com/wookie184/dupesearch/runs/2965153244?check_suite_focus=true (although I added a bit in the pastebin)
I'm getting maturin: command not found even though in the previous step poetry seems to install it correctly, • Installing maturin (0.10.6). I'm wondering how I can fix this, and also if i'm on the right lines in terms of what i'm doing in general, as this is all new to me.

tawny temple
#

Wouldn't poetry install that stuff to a venv? One that you're not activating

#

I believe there is a way to make poetry install to the normal site instead of a venv, but I don't remember

#

I'll try to find it

#

You just have to configure this to false https://python-poetry.org/docs/configuration/#virtualenvscreate

Configuration | Documentation | Poetry - Python dependency manageme...

Configuration Poetry can be configured via the config command (see more about its usage here) or directly in the config.toml file that will be automatically be created when you first run that command. This file can typically be found in one of the following directories:
macOS: ~/Library/Application Support/pypoetry Windows: C:\Users\AppData\Ro...

urban elbow
#

aha, that would make sense, i'll try that. Thanks

#

yay, that fixed it!

winter stream
#

So i want to make a Macro with python, that can simulate xbox controller inputs using python
After 5 hours of scouting the internet i am unable to find a working library for Ubuntu

#

Just point me in some direction

heavy knot
#

Hey! i would like to make a program who start me league of legend, and clique on "play", next to "ranked solo / duo" & then choose my role.
To open it's ok.
I don't understand how to :

  • put the screen of lol in 1st plan (if other's like discord has been launched)
  • how can i click at x: ... & y: ...

Thanks mates!

sharp totem
#

Hello, can someone recommend me a good docker course

burnt ibex
# sharp totem Hello, can someone recommend me a good docker course

https://youtu.be/pTFZFxd4hOI This video is really good for the basics

YouTube
Programming with Mosh
Docker Tutorial for Beginners [2021]

Docker Tutorial for Beginners - Learn Docker for a career in software or DevOps engineering. This Docker tutorial teaches you everything you need to get started.

Want to learn more from me? Check out the...

▶ Play video
sharp totem
#

Thanks

wide thicket
heavy knot
#

I'm trying to get github actions to auto-upload to PyPI if I tag the push

#

but using if: startsWith(github.ref, 'refs/tags') as part of the yml doesn't work even though a couple of sites say it should

#

my github.ref is "refs/heads/main", it doesn't start with refs/tags and I'm not sure why or how to make this work pithink

#

This is my yml ```yml
name: Build and Upload Python Package to TestPyPI and PyPI

on:
push:
branches:
- main

jobs:
publish:
runs-on: ubuntu-latest
steps:
- name: Check out repo
uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: Install wheel
run: python -m pip install wheel --user
- name: Install setuptools
run: python -m pip install setuptools --user
- name: Build a binary wheel and a source tarball
run: python setup.py sdist bdist_wheel
- name: Upload to TestPyPI
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: token
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
repository_url: https://test.pypi.org/legacy/
- name: Upload to PyPI
if: startsWith(github.ref, 'refs/tags') # <- doesn't work >:(
uses: pypa/gh-action-pypi-publish@release/v1
with:
user: token
password: ${{ secrets.PYPI_API_TOKEN }}

tawny temple
#

I've done this with Azure but not with GH Actions, so I'm not familiar with when the ref is a tag in GH Actions.

heavy knot
#

Ooh, I'll try that. I guess create will trigger whenever I push with a new tag aka version?

heavy knot
#

I think it worked discoYays just now I need two yamls, one for PyPI and another for TestPyPI

haughty finch
#

We have developed a small tool to export different variants from the same Python codebase and named it Python Variant Exporter 😆 .

It is useful, e.g., if you are a Blender/Houdini/Maya addon developer and want to export a free and a commercial version of an addon from the same codebase.

If you have code like this:

__VARIANT__ = 'DEV'

def example():
    if __VARIANT__ in ['DEV', 'PRO']:
        print('PRO')
    elif __VARIANT__ == 'FREE':
        print('FREE')

example()

Python Variant Exporter can export it for you to fit:

pve -d __VARIANT__=PRO example.py example_pro.py

__VARIANT__ = 'PRO'

def example():
    print('PRO')

example()

See here for more:

We are looking forward to your feedback and some battle testing!

rancid schoonerBOT
#

Hey @dire holly!

It looks like you tried to attach file type(s) that we do not allow (.pdf). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a.

Feel free to ask in #community-meta if you think this is a mistake.

rapid oxide
#

i need help flask

#

to deploy ML in android app

twin gust
#

Came across this line in GitHub Actions and I'm confused
What does that line do?
pip install .[lint]
what are . and []?

rich remnant
#

I imagine . is current folder and [] are basically defining the "optional extras" you can install from a package

#

haven't seen . before but it makes sense if interpreted like that, specifically for like github actions and the like

rancid schoonerBOT
#

setup.py line 24

extras_require={"azure": ["azure-servicebus>=0.50,<1"]},```
twin gust
#

Thank you

maiden shadow
#

Docker question.
I am looking to add this awesome web based VS Code server package to my docker image. : https://github.com/cdr/code-server
I currently have a very basic docker image that pulls Ubuntu and runs my flask app.
I want to add the code-server project so that I can access my code. How do I add this to my below image?

FROM ubuntu:latest

RUN apt-get -y update
RUN apt-get install python3 -y
RUN apt-get install python3-pip -y

WORKDIR /app
COPY . /app/

RUN pip3 install -r requirements.txt

EXPOSE 5000

ENTRYPOINT [ "python3" ]
CMD [ "app.py" ]
GitHub
cdr/code-server

VS Code in the browser. Contribute to cdr/code-server development by creating an account on GitHub.

tawny temple
#

It looks like they already have an image they publish on Docker hub. What are you trying to accomplish by adding it to an image that has flask? Is there a reason these images need to be combined rather than running as separate containers?

maiden shadow
#

Confession: I'm a docker rookie.... I was hoping to add them both to the same image. does that make sense?

tawny temple
#

No, it generally doesn't make sense. An image should generally be focused on running one thing.

#

You can have one container for your flask server and another for the code server. You can make it easier to start them together by using docker compose.

maiden shadow
#

Make sense. Thanks.

rustic hill
#

hey guys anyone knows about RH openshift? Is it possible to create a ubuntu image for my application instead of RH linux?

#

In my dockerfile even if FROM ubuntu:bionic is used it does not create ubuntu

rich remnant
#

openshift is just rebranded kubernetes

#

if FROM ubuntu:bionic does not do what you think it does, then you either expect the wrong thing, or someone has messed around with your Docker host beyond just it having openshift

rustic hill
#

am i not understanding it correct? FROM ubuntu:bionic does not create an ubuntu image?

unknown.png
floral smelt
#

Not sure if this is the right channel for it, but has anybody got good examples of CLIs done with Python and the click package?

#

I'm trying to understand how I can personalize my cli with the namesake and the different options (ie. mycli read -f=myfile.txt)

wooden ibex
#

Containers just layer on top of host OS

#

So red hat release file will still be there because host OS is Red Hat

rustic hill
#

😮 sorry, i shifted from aks to aro and its still confusing to me sometimes

wooden ibex
#

This isn’t Kubernetes thing, this is container thing

rustic hill
#

So whenever I deploy a new app, its host os will stay as rh?

wooden ibex
#

Depends on host OS

rustic hill
#

Ok i see

#

So a new app is a container?

brisk lake
#

Docker networking issues

networks:
  frontend: {}
  backend: {}

services:
  site:
    networks:
      - frontend
    ports:
       - 3000:3000
  server:
    networks:
      - backend
      - frontend
    ports:
      - 5000:5000

And then in my site package.json i have

"proxy": "http://backend:5000/"
#

But when i fetch "/" from my frontend i get back the frontend's html

#

Frontend is a CRA app with just one component to make this request and backend only has a route / thats supposed to return {"hello": "world"}

#

Also i can curl each container with a proper response from outside but not the backend from inside the frontend

heavy knot
#

i wanna ask input("whats ur name:") and then put print i like the name and put whatever was put in the input

foggy reef
#

i want to make as light a venv (using the built in venv module) as possible,

  • how do i get the --symlinks argument to work? on windows i keep getting a bunch of errors like Unable to symlink 'C:\\Users\\admin\\AppData\\Local\\Programs\\Python\\Python39\\python.exe' to 'C:\\Users\\admin\\Documents\\Projects\\venv-lite\\.venv-without-pip-symlinked\\Scripts\\python.exe' any time i use it.

  • is there a way to have it symlink pip and its dependencies into the venv instead of creating an entirely new copy of it?

urban goblet
#

@rancid ingot pipenv

rancid ingot
#

Yes

#

How's it compare to pyenv + poetry

urban goblet
#

Not sure, haven't used Poetry

#

pipenv is a first part tool I believe

#

Going to a Python meet tonight so I could ask

tawny temple
#

It is not first-party. In fact, there isn't any first-party package management tool.

#

pyenv can be used with pipenv too. The actual comparison would be poetry vs pipenv.

#

Poetry installs dev dependencies by default. Pipenv does not.
Pipenv can be configured to inline shell scripts (like for dev utilities). Poetry can only run Python scripts.
Poetry has a plugin system in development.
By some accounts, poetry is faster to lock and install dependencies.
Pipenv has had a bump road with maintainership, but they might be getting past that by now.

#

Poetry uses pyproject.toml. Pipenv has its own toml config file.

#

Poetry can be used to build packages too. Pipenv cannot.

velvet granite
#

Any idea why my buildozer gotten this error?

Original exception:
===================

    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/dist-packages/sh-1.14.2-py3.7.egg/sh.py", line 2071, in __init__
        tty.setraw(self._stdout_child_fd)
      File "/usr/lib/python3.7/tty.py", line 28, in setraw
        tcsetattr(fd, when, mode)
    termios.error: (13, 'Permission denied')

What permission is it needing..? I may have missed something

#

I did a chmod on sh.py but same thing

nimble niche
#

Hi all,
I made (yet) another Python profiler tool (can be used like cProfile, or with manual instrumentation, and the data are visualized in real-time in the viewer): https://github.com/dfeneyrou/palanteer
It is high performance, low dependencies, provides a lot of different views (timeline per threads, memory timeline, plot & histograms...) and automatically collects functions (Python & C calls), garbage collection runs, exceptions, OS memory usage...)
The last added feature is the support of asyncio / gevent. Visualization of such async execution is really helpful to understand what happens under the hood.

GitHub
dfeneyrou/palanteer

High performance visual profiler, debugger, tests enabler for C++ and Python - dfeneyrou/palanteer

#

Feedbacks are welcome!

urban goblet
tawdry needle
#

In a Pycharm project with multiple attached "sub-projects", is it possible to open multiple Python Console instances, one for each sub-project?

urban pecan
#

hey @tawdry needle just the one i was looking for 🙂

#

maybe you have an idea how to solve that problem

tawdry needle
urban pecan
#

we've mostly figured out how to solve the installation problems with libs like numpy in justuse, so the next step would be to get stuff working that depends on a specific version of numpy

#

now, my idea was to have an argument like use("pandas", auto_install.., import_to_use={"numpy": {version: "1.23.2", hash_value..}})

#

somehow intercepts imports and transform those calls into calls to use() with the given arguments

#

any idea how to do that or a better way to deal with specific version-dependencies?

#

if we only version-pin pandas, and it would try to import numpy, it could only be the one installed by pip, but not a specific version

#

and you probably know dependency hell just as well as i do ^^

tawdry needle
#

i had suggested creating a new site-packages directory for every top-level use invocation

#

then you can have a full isolated tree of dependencies, npm-style

urban pecan
#

well, we do have a completely isolated packages folder

#

but the problem is that code that is calling to import would look for sys.path

#

so we'd either need to do tricks on sys.path or on import to look elsewhere

tawdry needle
#

yeah, i had suggested prepending the new site packages directory to sys.path

#

i think that would work, right? while use('pandas') is executing, all import numpys would be resolved w/ the temporary site-packages directory at the front of sys.path

#

however this might be incompatible with the auto-reloading... you'd probably have to go even deeper into import machinery for that, or end up with some awful locking around sys.path for thread safety

#

as for the "import to use" specification, maybe start by mimicking what setuptools.setup uses? put hash as a separate argument maybe

urban pecan
#

could a wrapper around __import__ maybe work to intercept calls?

tawdry needle
#

to intercept imports? i have no idea

urban pecan
#

yeah.. that might be the simplest solution

#

if it works

#

in principle, not that code as a dependency, i mean

tawdry needle
#

you know more than me 😛

#

i doubt you could use this as-is, but you could probably reuse its guts

urban pecan
#

hmm.. i'll try, although i don't have a good feeling about messing with meta_path, might end up having the same problems in recursive cases as with sys.path

#

thanks

tawdry needle
#

i'd make auto-reload and isolated deps mutually exclusive tbh

#

auto-reload is only good for local dev, right? you should have a venv or something for that anyway

short peak
#

what do you guys usually use for auto-reload?

urban pecan
#

you only can auto-reload single modules

#

auto-install only applies to packages (use() with str, not a Path)

#

no point in reloading packages that should be hash-pinned anyway ^^

#

same goes with use()-ing an URL

#

no reload there, since the content should never be able to change

#

not sure yet whether or not that might change if we go for crypto-signed code instead of hash-pinned, but for now it's all locked down

#

i'm not really eager to get into crypto-signature stuff.. probably will need to find a security specialist to help me get this right

#

cryptography certainly is an area where be dragons

#

and i need better armor before i can tackle those 😉

wooden ibex
urban pecan
#

aww ❤️

urban pecan
#

oh, wow, that might be easier than expected 😄

# 
from importlib import __import__
import builtins

def wrapper(func):
    def decorator(*args, **kwargs):
        print("wrapped!")
        return func(*args, **kwargs)
    return decorator

builtins.__import__ = wrapper(__import__)
import math

=> wrapped!
coarse solar
#

Hey guys I am looking for a program that will create a random character based on png. files that I have anyone have a program lying around that could do this?

bright cipher
#

Hi is there any module to work on accessibility PDF or else autoaged PDF document to get tags bounded inside the document.

#

I need to retrieve boundary boxes of each tag bounded to PDF document.

gleaming portal
#

http://pydepsearch.dwrodri.com

pip search was down and I couldn't sleep so I made a JSON API for searching PyPI project names. API endpoint is /search and the param is query

the entire client is right here:

import sys
import requests
from urllib.parse import quote as encode

for url in requests.get(
    "http://pydepsearch.dwrodri.com/search/",
    params={"query": encode(" ".join(sys.argv[1:]))},
).json()["results"]:
    print(url)
#

of course I get random timeouts

#

now that I start sharing the URL

sick ravine
#

has anyone done Apple sign in before, how do you prevent double account, when the user does sign up with email and password, but later want to now use sign up with apple

heavy knot
#

@sick ravine You'd probably add Apple as and optional sign in option. I donno, I haven't done anything like that

gleaming portal
#

fixed

sick ravine
#

@heavy knot thanks for the insight

heavy knot
#

😀

urban pecan
#

create a random character based on png could mean anything from OCR to hashing and pseudo-randomness

fervent orchid
#

Hey not sure if this is the right spot, but is there a way to set up NixOS in such a way that you can declare package use for a given virtual environment in python?

wintry iris
#

So I'm using vscode and am attempting to commit to github. I can do this, but some files I do not wish to commit such as my workspace file are being staged for commit as well. How can I make sure this does not occur? I heard of something called a gitignore, is this what I need to look into?

burnt ibex
wintry iris
burnt ibex
wintry iris
#

thank you very much sir

sacred thistle
#

any idea about EC2 ?
what i know is its a instance like VM ?

wooden ibex
#

it's a VM

broken field
#

I'm developing a CLI using argparse and I want it to be able to be installed with a Windows executable file, no Python needed, and then be used normally in the console. I'll use pyinstaller but which script of my project should I freeze? setup.py?

#

or should I use freeze the main file which parses the arguments and then just call the app as myapp.exe h?

broken field
stuck tide
#

Apologies if this is not the appropriate channel for my question.
I am creating a pip package, and it almost works, but the entry point is giving me weird problems.
If I pip install, I can navigate to the module folder in site-packages and run it properly with python3 main.py
If I try to run it with my entry point, the program will run, but some parts won't work. For example, main.py imports some other files into it, and it looks like everything it imports from one particular file is just not working at all. It's hard for me to go more in-depth without going far in detail.
My entry point is set up like this in setup.py:

   { "console_scripts":
       [
           "run_simulation=<module_name>.main:main",
       ]
   }```
where <module_name> is the name of the module.
#

I should also note that in main.py, I have if __name__ == '__main__': main() and nothing else inside that if statement. So I believe running it manually and using the entry point should give the same results, correct?

#

I just tested this, and I think I found the problem.
If I use a separate file to run import main main.main()
Then the program will run, but with those weird errors I was getting with the entry point. However, if I do this: from main import * main() Then the program will run perfectly. Why is this the case? Does anyone know a way I can fix this?

tawdry needle
#

@stuck tide show your file structure, and the rest of your setup.py

stuck tide
#

File Structure:

  • pkg_name
    • setup.py
    • README
    • module_name
      • main.py
      • __init__.py
      • ... (other .py files imported to main)

setup.py:


setup(
    name="pkg_name",
    version="0.x.x",
    packages=["module_name",],    # name of modules
    package_data={
        "module_name": [
            "Models/*.obj",
            "Models/*.png",
            "Models/Notation/*.png",
            "hints/*.png"
        ],
    },
    include_package_data=True,    # make sure resources are included
    url="https://github.com/...",
    # license
    author="mandrew",
    # author_email
    description="Rubik's cube simulation and tutorial",
    install_requires=["ursina", "rubik-cube", "psd-tools3"],

    entry_points = 
    { "console_scripts":
        [
            "run_simulation=module_name.main:main",
            #"command_name_2 = src.file_name:func_name"
        ]
    }
)```
stuck tide
tawdry needle
#

happy to help w/ specific questions. but rather than repeat myself, i think the message i linked should explain fix your file structure

stuck tide
#

Thank you for the link; I think my file structure is already following what you said in that message. My setup.py is in the directory above the module being distributed, next to readme.

#

Oh I'm sorry I didn't read far enough yet

#

So all of my imports need to have the module name in them. I'll go give that a try

#

Okay, I read the full message and I think all of my code already satisfies what is outlined there. I checked my import statements and they are all in terms of the module name

tawdry needle
#

@stuck tide in that case, you might want to use packages=find_packages() instead of packages=["module_name"] -- packages=["module_name"] will not infer e.g module_name.main

#

also, you can use module_name/__main__.py instead of module_name/main.py. then you can invoke your tool with python3 -m module_name, and your entry point would be updated to run_simulation=module_name.__main__:main

#

i'd also strongly suggest adding a pyproject.toml file alongside setup.py, containing

[build-system]
build-backend = 'setuptools.build_meta'
requires = ['setuptools >= 51.0.0', 'wheel']

you can use an older version of setuptools if you need to e.g. support py 3.5

stuck tide
#

Oh yes, I've been meaning to add the pyproject.toml file I just wanted to get this part working first. Will give __main__ a try, as well as find_packages(). Thanks!

#

Thanks so much! Using __main__ and python3 -m module_name worked a treat 🙂
I may continue trying to get the entry points to work with other things, but I'm really just glad this was able to get it working. Thanks again for your help & time!

storm ice
#

Hello world

#

I need to make an app for android, that will probably require a lot of access to phone's functions (like making calls). I don't have a clue where to start, but I would like to avoid going to android studio or anything else in the Java/kotlin field (as I'd be starting from zero). Could I use my python experience somehow?

halcyon needle
rustic hill
#

hi guys, a question on aro, after pulling image from container repository, apply deployment.yml how can i expose the service ?

unknown.png
burnt thunder
#

is there some way to customise the pyenv installation process/have it use a custom executable, I need a debug build of python with valgrind and no pymalloc and would rather not write the shim myself

tawny temple
burnt thunder
#

ah, that seems useful, thanks

woven saffron
#

I see these fancy coverage badges everywhere, how do I get one of these for my private gitlab project?

#

(I presume they are created dynamically for every commit?)

burnt ibex
woven saffron
#

Very nice thanks

rugged flame
#

Maybe try following that guide, and ask here if you get stuck on anything.

#

If you install Homebrew natively, it will be installed into /opt/Homebrew; if you install it with Rosetta, it will be installed into /usr/local.

#

It's all a little bit involved, but once you have it up and running you shouldn't have to worry about it (hopefully) lemon_sweat

wary oasis
#

Has anyone here had issues with VScode's intellisense not working properly pygame? It will recognize thing like pygame.Surface() or pygame.Vector2() but something like pygame.time will not show up in code completion unless imported explicitly ```py
import pygame

pygame.display.set_mode()
doesn't workpy
from pygame import display

display.set_mode()

rapid sparrow
wary oasis
#

Hmm that didn't work, thanks though.

#

I'm just gonna download Pycharm, goodbye memory 😿

rapid sparrow
#

console debug with pdb works fine though

#

I guess I should use different firefox version to fix that. Some sort of imcompatibility in the latest release

#

or perhaps to try updating firefox

heavy knot
#

Anyone know a website/tool that will order and consolidate links in a markdown file? MHXThink

#

I could swear the Stack Exchange question textbox used to do it rooHmm

wooden ibex
#

@deep estuary what ingress are y’all using?

deep estuary
#

we use traefik because:

  • dead simple mTLS
  • (used to have) dead simple TLS (we now do cert-manager so we run traefik in HA)
  • nice middlewares
  • simple deployment
#

I'm considering a move (well, we almost certainly will) to nginx-ingress, because personally I'm more fond of nginx and it's much more configurable, we're not using the TLS features in Traefik that we selected it for and it has better integration with the ingress resource

wooden ibex
#

Ok, I’ll look at both

wooden ibex
#

I also don’t want to deal with Helm for simple stuff

tawny temple
fierce swan
#

I seem to have a weird issue where when I updated discord.py-stubs, they seemed to break

unknown.png
fierce swan
#

solution, uninstall the stubs and reinstall them again

obtuse saddle
#

Two questions:
How do I run "external tool" in Pycharm with project's venv (conda) activated? It works perfectly via terminal, but I can't find a way to properly activate it. There's conda run, but it's "experimental".

How can I run code in external terminal? I know about "terminal emulation", but some advanced features of TUI frameworks work in dedicated terminal, but not in Pycharm's one.

fallow gulch
#

how can I return the single like of the list?

unknown.png
urban pecan
#

@fallow gulch what?

fallow gulch
#

is fine i got it

oak hull
#

does anybody here know stuff about barcodes and article-names? 🙂

#

this is prolly not a question for this channel but

#

pewpew

rare sundial
#

I have a 3D plot created with matplotlib. I want all axes to have the same scale. However, setting the axis to 'square' does not seem to work in 3D. Also, as the plot is supposed to be interactive (the user can zoom in/out and rotate it), matplotlib cannot really cut it. It is too slow to be of any use. Therefore I tried plotly, which seems very powerful. However same problem, how can I force it to use the same scale for the x, y and z axis?

warm furnace
#

guys, is poetry compatible with python 3.10?

#

3.10.0b3 specifically speaking

night quest
#

Can you recommend any books to mastering git? I know some basic features like merging, cherry-picking, rebasing and so on but I think it's only the beginning so any books recommendations are nice to see!

rare sundial
night quest
oblique grail
#

good night, i wanted to create a plugin module in jpython for the autopsy tool. fetch data from SQLLITE databases.

dire reef
#

Hi, I see tox used for automated testing in a lot of packages on github but not all of them seem to be state of the art and some seem to feature some boilerplate code. Therefore, I wonder whether a package developed from scratch should also implement tox or whether it is simpler to test the package for example with pipenv and github actions (if the project lives on github anyway). Put simply: What is a state of the art CI workflow for packages and does tox play a role?

west sluice
#

what's the best way to install the latest version of tensorflow-gpu on linux?

#

anaconda has v2.5.0 for windows but not for linux 🤔

#

and conda-forge doesn't even have tensorflow-gpu

#

I'd rather not use pip if possible because I need the cuda and cuDNN toolkits pulled in too

tawdry needle
#

@west sluice don't use the anaconda repository, use conda-forge

#

oh, conda forge doesn't have it? weird

#

huh

#

pip install tensorflow-gpu

#

or better yet, the regular pip install tensorflow already has gpu support

#

not sure why anaconda hasn't updated their package yet

west sluice
#

hmm... I'll probably have to conda install cuda and cuDNN if I pip install tensorflow

tawdry needle
#

yeah it might "just work" if you do it all in the same conda env

west sluice
tawdry needle
#

that i do not know

#

it looks like tensorflow only "officially" supports pip install and not conda

#

then again they also say "ubuntu" and not "linux"

#

no idea what exactly that means

west sluice
#

which sucks, since there is a linux 2.5.0 package there on anaconda

#

maybe I should look at the docker image?

#

oof, the tensorflow docker images all use python3.6... it would be nice if they used a more recent version of python

#

😭 why is this so hard?

tawdry needle
#

what in the world

#

so weird

#

py 3.6??

gleaming portal
#

anyone ever encounter issues with external index URLs insisting on downloading and installing the oldest wheel?

next scroll
#

I have a GitHub workflow that pushes a Docker image to ghcr. The docker image has a label LABEL org.opencontainers.image.source=https://github.com/[...]/[...] and the package is being pushed to the same path on ghcr, however the package is not appearing on the right bar in the repo under packages

#

I'm trying to avoid having to manually link it via the UI, as this is for work and only 1 person has organisation owner perms, and requires a lengthy change request each time

safe mango
#

hello

brisk lake
#

is it possible to tel poetry to publish a package under a different name than the project name?

#

i just realised the name i have is taken

cloud elbow
#

[vscode]

Why does not this workspace configuration work?

I'm using multi-root workspace, so I need to configure different terminal working directory for each project

this is the config

"folders": [
        {
            "path": "facilities",
            "name": "facilities",
            "pythonPath": "facilities/virtualenv/bin/python3",
            "terminal.integrated.cwd": "facilities/clone", // why does not this work?
        },
]

also asked in #editors-ides but got no answer

velvet spire
#

hey all, i 'm having trouble configuring gpg to sign commits on windows, Ik that @stable cloak does it but it keeps telling me that a key does not exist, when its the same thing as it is on my linux installs.

I have git 2.32.0.windows.2 installed, with gpg version 2.2.28

#

what the

finite fulcrum
#

Are you using git's gpg?

velvet spire
finite fulcrum
#

newer git versions come with their own gpg which should make it easier as you don't have to download it yourself

velvet spire
#

oh is that the problem here

#

how do i use that gpg version?

#

because i think that's the problem

#

git is trying to use its own gpg

#

on linux its not a problem, it uses the system gpg afaict

finite fulcrum
#

Did you configure gpg.program in git's config?

velvet spire
#

i changed it to gpg

finite fulcrum
#

I recall having problems with the 3rd party gpg distributions for win where they were using some weird paths so maybe that could be it, but it has been a while

#

I don't think it should be using the wrong one if you configured it, but there's also no reason to use that when git comes with its own. git has it under .../git/usr/bin/gpg.exe

velvet spire
#

ah.

#

so that's which one it was using

#

ig i'll reinstall the git for windows stuff

#

and change the configuration to be the full path to it

velvet spire
#

@finite fulcrum so after reconfiguring it to use the other one, i think it is using the proper one now, however it has a few problems

WindowsTerminal_uvwujDGcVP.png
finite fulcrum
#

is that trying to start --show-source

#

was that a clean install?

velvet spire
finite fulcrum
#

The only thing --show-source came up with was a flag for flake8 so I've got not clue how it got to that

#

Does commiting, cloning, etc. work without the signing?

velvet spire
#

AH IT WORKS NOW

#

thank you @finite fulcrum

velvet spire
#

i cleared it from the local settings and now it works properly finally

#

in summary, the solution was basically changing the gpg.program to the path of the actual gpg client since on windows git brings its own gpg agent

rapid sparrow
#

It is possible having something like haproxy
but instead of sending requests from same source to different ip destinations
it would send them through different available network interfaces?

analog pelican
#

Hiya, tldr:
Firewall on - wsl network unreachable. Turning off firewall- everything is perfect. how to fix?

does anyone know how can I find out my wsl port number? im trying to make a rule for it, so that firewall wont block the network there, but I dont know how can I find the port number 😬

stuck tide
#

@tawdry needle it's been a while, but you gave me some advice on starting my project by using python3 -m module_name since my entry points weren't working (and this worked a treat). I wanted to let you know that I looked at some documentation and I found that using the scripts keyword rather than console_scripts in setup.py ended up getting my original run command working.

scripts allows the user to write an actual script, rather than just run a single function. Doing this allowed me to add an import statement before the function call, and that ended up working.
Here's the link to the documentation if anyone's interested: https://python-packaging.readthedocs.io/en/latest/command-line-scripts.html

tawdry needle
#

good find and glad you got it working how you wanted it

#

i actually didnt know about scripts

#

note that this doc is somewhat out of date, e.g. tests_require is deprecated by setuptools

supple venture
#

What I am doing wrong here?

Screenshot_from_2021-07-15_11-33-08.png
#

i do copy everything from . to . so it should have worked

rapid sparrow
#

pyppeter is a pervert application

#

i used page.evaluate('(element) => element.value=""', elem[0]) in order to clear input box

#

selenium with elem.clear() is much... simpler

#

well, pyppeter has more functionality though

brisk lake
#

anyone use https://github.com/nektos/act or can suggest any alternative ways of locally testing a CI action?

GitHub
nektos/act

Run your GitHub Actions locally 🚀. Contribute to nektos/act development by creating an account on GitHub.

opaque gorge
#

I have 2 separate git branches in my repository with 1st being the original default branch I had and 2nd being a branch for different distribution, it has a completely different src folder, which I don't want to merge with the 1st one, but I do want to keep the static folder in sync, how could I merge the 2nd branch into the 1st one without merging some changes, I thought I could somehow resolve it in a git conflict, but there was no conflict because I've just been developing on 2nd one and syncing the 1st one into it since I was only doing static changes there, now that I did some static changes in the 2nd branch I can't sync them in the 1st one because it changes the src file which i don't want

tawny temple
#

I don't think branches are meant for this sort of thing. The problem is that the history of the 2nd branch has commits that change the src, and if you try to merge, it will include those changes. You would have to edit all the commits and change the history to not include changes to src, or you would have to merge it in and then make a new commit to revert to the original src. The former would be very tedious and the latter would make for a messy commit history.

opaque gorge
#

hm, yeah, I figured it wasn't an ideal solution to use 2 branches like this, but I didn't want 2 separate repos for something that will share many aspects of the code

tawny temple
#

You could just copy over the static folder manually and make a single commit for it. it would be very easy to do, but of course you lose all of the git history/blame for the new changes.

#

I don't have the full picture, but I would have tried to keep it all in one branch and have two separate directories for distribution 1 and distribution 2.

#

Which only contain things that are unique to those distributions.

opaque gorge
#

well it's not completely that easy since even the static folder isn't completely equal, just for the most part, there are some minor differences based on the distributions and copying content that's mostly the same into 2 folders and copying things whenever I change something in the first would be very tedious. there are also some share aspects with the config just not everything, I was hoping there would be something like .mergeignore file just like there's gitignore where I would just define files that shouldn't be merged or something similar

tawny temple
#

Could you create a script or tool that sorts out these differences when it builds/packages the distributions?

opaque gorge
#

I don't think so, I still need a way to get the edits done on one branch somewhat merged onto the other branch, I just don't want certain things merged, building is then only done from one branch, there's way too many differences to simply resolve this in some script

tawny temple
#

So there's a --no-commit option for git merge apparently

#

You can edit the merge and unstage the files you don't want changed

#

Furthermore, you might have something similar to a .mergeignore with .gitattributes

#

Some interesting stuff that I never came across before, cause this is sort of a niche use case.

opaque gorge
#

hmm, I'll have to experiment with these a bit and see how they work, thanks for the suggestions

obsidian blade
#

I have an issue, i cannot use modules unless i take them to the same directory/folder as the python file, is there a way to fix that? Because otherwise it would say "No module named..."

arctic shale
#

I am trying to install docker desktop

#

and it said install stuff for WSL 2

#

should i tick

#

i don't want WSL tho

#

will it work fine if i just untick that box

#

and use docker without WSL 2

vocal pewter
#

If you don't use WSL then you don't need to enable the feature. It's there so that you don't need to install yet another docker in your WSL if you use one.

tawdry needle
arctic shale
#

can i do it properly

#

the setup

vocal pewter
arctic shale
#

yep

#

alr

#

thanks

vocal pewter
#

I tore my setup down and rebuilt it at least six time in a day trying to work with a company vpn. Didn't seem to mind me adding/removing WSL every other reboot darkoLUL

wooden ibex
#

Which is very unusual setup

vocal pewter
wooden ibex
arctic shale
#

oop

#

alr thanks or letting me kn

heavy knot
tawny temple
#

Wow there's a blog post for everything

#

If you look hard enough

heavy knot
#

Playing around with it I learned markdown reference links don't need extra [] if they are their own label ```md
This link has its own label so it works like normal.

#

Which complicates the idea of renaming all the reference link labels Thonk

cobalt cargo
#

Maybe a random Q…. But is there a way to use Jupyter on a iPad?

wet shadow
velvet spire
#

what the fuck

#

github cli is insanely overpowered

#

i have it streaming a livefeed of a current github actions run

rugged flame
#

Maybe this is a stupid question, but what actually is "vendorizing"? 😄

velvet spire
#

what context?

#

I've marked files as vendorized before in .gitattributes

finite fulcrum
#

If you mean vendorizing dependencies then from what I understand you take a known working version of a dependency and include that in your application (in an isolated manner).
For example pip has the vendorizes requests (and it's dependencies and maybe some others). So if you have pip it has its own requests that are independent of what you'd install through it and only gets updated manually

haughty tiger
#

can someone help me with this error I'm getting from a project with uvicorn?

...
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 846, in exec_module
  File "<frozen importlib._bootstrap_external>", line 983, in get_code
  File "<frozen importlib._bootstrap_external>", line 913, in source_to_code
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
ValueError: source code string cannot contain null bytes
#

source code string cannot contain null bytes??

#

tried deleting and recreating the virtualenv, same error...

#

is this even the right place to be asking lol

tawdry needle
#

@haughty tiger how are you invoking it? Sounds like somehow you ended up with a null byte (the byte 00000000) in your code

#

Usually you aren't supposed to have that in text, a lot of programs don't like that

haughty tiger
supple venture
#

could you share your code, it might help

tawdry needle
#

Is there more to that traceback

ebon spire
#

Hi, I have the same problem with this one from SO about git "would clobber existing tag" https://stackoverflow.com/questions/58031165/how-to-get-rid-of-would-clobber-existing-tag. Though it works to me the git fetch --tags -f solution. The SO thread doesn't give much explanation why that message/warning show up every time I do git pull. I hope someone can help me explain about that message and how such git fetc --tags -f works as a solution.

Stack Overflow
How to get rid of "would clobber existing tag"

I'm using git in VSCodium and each time I try to pull git is complaining.

Looking into the log I see

git pull --tags origin master
From https://github.com/MY/REPO

  • branch master...
tawdry needle
#

What error message do you get in your case?

#

Is the maintainer of the repository continuously force updating a tag?

#

Tags aren't meant to move around, that's what branches are for

haughty tiger
tawdry needle
#

how the heck did you get null bytes in there in the first place

haughty tiger
#

your guess is as good as mine....

#

not even the slighest clue haha

#

maybe something did behind my back in vim or emacs??
but usually they are good about showing non printable characters

#

yep, it's fixed now

#

how unbelievably odd

heavy knot
#

anyone know how to take off or make this pop up smaller in atom. it block basically my entire screen when i hover over something. the pop up that you see at the top of the window btw

Screenshot_from_2021-07-18_14-20-16.png
velvet spire
#

How to remake commits since I made them but they aren't signed

#

I want to remake the commits but just sign them

vague silo
#

git commit -S --amend for the last commit

#

for anything else, you'll need to rebase

velvet spire
#

I don't want to change anything else about them

vague silo
#

IIRC if you just run a rebase without making any edits it will re-make the commits (e.g. git rebase -f HEAD~5)

tawdry needle
#

☝️ exactly that. it's not unlike cherry picking the commits in order

#

basically copying their contents but with new hashes

#

in fact i think rebase --interactive uses cherry-pick internally, while rebase non-interactive uses am i think

velvet spire
#

Make a new branch from the current one, git reset HEAD~7 --hard

#

And then git rebase -S other branch

heavy knot
#

Is it possible to pass GitHub repo secrets to K8s cluster through Github Actions?
I'm using GKE atm but looking for a cloud-agnostic solution to do so. Using Kubectl with Kustomize

vale wedge
#

anyone aware of a openapi v3/3.1 capable client library for python that can make the models/api objects from a spec, ideally also supporting deep object encoding

wooden ibex
heavy knot
#

Now just need to figure out how to use the multibase kustomization to use different secrets

mortal zodiac
#

Hi all, I need some help with doing a Pull Request

#

On github and git from terminal

#

I have a branch called add_feature

#

...and in the past have pushed that branch to Github to do a PR with no problem

#

I tried doing a push today one the same branch to the same PR but getting some errors I havent seen before

#

When I do git log I can see my commit is the HEAD

#

One thing that might have screwed me is that I didn't pull from the beta branch first before trying to push my changes

#

I tried to do a pull but getting:

   error: please commit or stash them```
#

Any thoughts?

#

Also when I do git status I don't see my tracked files I added. I see other files I don't want to commit.

vague silo
#

@mortal zodiac for the most simple case, about that cannot pull with rebase error, you can try

git pull --rebase --autostash

that will automatically stash away your changes before pulling. that said, if your local work conflicts with remote work, that might throw up. as for other files - if you don't want to commit them, don't add them with git add, and if you are 100% sure that you do not need the untracked files, use git clean

velvet spire
#

@knotty frigate i'm using starship

maiden prawn
#

pure

#

zsh

#

i mean pure named prompt, not that my zsh is pure

heavy knot
#

I have the following directory structure (cwd is parent folder of deployments):

..
deployments
   ├───kustomization.yaml
   │
   ├───beta
   │     ├───deployment.yaml
   │     ├───kustomization.yaml
   │
   └───latest
         ├───deployment.yaml
         ├───kustomization.yaml

Now I want to build and rollout either the beta or the latest one based on an env flag.
Till this step, I have no issue getting to work. But here comes the challenge.
I am using kubectl to create a generic secret from literals. The secrets data and the name will be automatically generated based on this env flag. So the secret created will either be secrets-beta or secrets-latest.
Now how can I consume secrets-beta inside beta/deployment.yaml and secrets-latest inside latest/deployment.yaml?
I am using envFrom and secretRef to convert those secrets into env vars for the container

wooden ibex
heavy knot
#

What kinda script? I'm currently running this as part of a GitHub Action

wooden ibex
#

Powershell or bash should be able to handle it

heavy knot
#

okay

#

Disclaimer: This is my first time dealing with K8s so I'm very green at this

wooden ibex
#

I use Powershell but I’m bias

#

Have fun

heavy knot
#

So what exactly do I make this script do though?

#

Like, use the kustomize edit command or something?

limber dome
#

this is great

keen reef
#

anyone uses pillow image grab? if so whats the parameters for box = (1,2,3,4) which is height /position/length?

keen reef
wooden ibex
heavy knot
#

Nah, that part I figured out long ago. The issue was with passing the secrets. But I figured that one out too, it was just a matter of switching the working dir

tame lodge
#

I want to host my discord bot on vultr, is there anything i can refer to? Ive already seen a documentation but its in js, i want a python version.

topaz dove
#

Hey I'm deploying some python services using docker-compose. Some of the services share the same python modules and today I'm making copies of the module to place it within the build context of the services where it's used. However, this causes repetition which is hard/annoying to maintain.

Are there any recommendations/preferences on how to share just one file to multiple docker services? From what I understand, fetching files from outside the build context, e.g., COPY ../../my_module.py in the Dockerfile is not allowed.

Please @ me.

rapid sparrow
#

at least it can work as a compromise

#

launch with make build

heavy knot
#

ive tried following the steps but keep get this error

#

online it doesnt mention anything about bios, just says to enable hyper v for windows

urban elbow
#

Have you enabled virtualization in your BIOS?

heavy knot
#

firmware is bios?

severe flame
#

bios is firmware

#

you can search online where the option is located for virtualization, providing your pc model and bios version

urban elbow
#

If you open task manager and click on CPU, do you see any mention of `Virtualization" in the details? Also that ^

#

some processors may not support it though, I think

heavy knot
#

i see, well let me try reboot pc and load bios

urban elbow
#

I have a question about a python + rust project I've made, (https://github.com/wookie184/dupesearch). I'm using poetry and maturin, but am struggling to get them to work well together nicely

My current build/publish workflow (https://github.com/wookie184/dupesearch/blob/master/.github/workflows/python-publish.yml) doesn't use poetry at all, as maturin seems to handle installing any dependencies, however for local dev on the Python side i'd like poetry (for creating the virtual environment, and managing dev-depencencies, for example).

However, I now feel like i'm having repeat lots of information for both systems, for example I list all project information and deps in my cargo.toml (https://github.com/wookie184/dupesearch/blob/master/Cargo.toml) and i've also ended up listing it again (and the deps twice!) in my pyproject.toml (https://github.com/wookie184/dupesearch/blob/master/pyproject.toml).

Is there a way I can get poetry and maturin working without repeating myself? If yes, are there any projects that do this I could take a look at, and if not, what would a good alternative be?

supple venture
urban elbow
#

I think maturin only supports it under the [project] section as a list under dependencies, like

[project]
dependencies = ["click^=8.0.1", "rich^=10.4.0"]

whereas poetry only supports it under [tool.poetry.dependencies], like

[tool.poetry.dependencies]
python = "^3.7"
rich = "^10.4.0"
click = "^8.0.1"

I was sort of wondering if there was a way of merging them together, but I don't think there is

#

As long as poetry's plugin interface is not finished and there is not standardized way to specify dependencies (https://discuss.python.org/t/pep-621-how-to-specify-dependencies/4599), I expect that you will have to duplicate at least the dependencies between poetry and any other tool, maybe even other metadata (until PEP 621 is widely used) and won't be able to use poetry publish.
iirc poetry's plugin interface was released recently, at least into beta, not exactly sure what the dependencies problem is, but hopefully there'll be a better way of doing this in the future 🤞

urban elbow
urban elbow
#

hmm actually i might just be able to use maturin by itself

#

ahhhh i think I was just overcomplicating things

heavy knot
#

If I have a deployment.yaml spec like this:

  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: app
        image: gcr.io/PROJECT_ID/IMAGE:TAG
        imagePullPolicy: Always
        envFrom:
        - secretRef:
            name: SECRET_NAME

I can use kustomize edit set image gcr.io/PROJECT_ID/IMAGE:TAG=gcr.io/$PROJECT_ID/$IMAGE:$TAG to build them with actual values.
Is it possible to somehow accomplish the same for SECRET_NAME?

stray pagoda
#

guys, any have o web site when i can make ssl certificate ?

buoyant bison
#

hey
does someone here use asdf for version management? 🤔
for node, for example

haughty finch
#

We just published our new Python Distribution Tools, which are most useful for Python addons/plugins/extensions. Ready to be battle-tested.

Python Vendoring Tool - Vendor libraries in a subpackage, which can be placed anywhere in a project.

Python Variant Exporter - Export a specific variant from a codebase.

Python Packer Tool - Pack a package into a single Python file.

Have a look here: https://pypi.org/project/pdistx/

PyPI
pdistx

A toolset for distributing Python projects in a convenient way.

merry patio
#

Is there a good way to have black format all incoming PRs to a certain branch?

cloud prawn
#

Git/Github related. So if I want to keep all of the versions as the project builds up, do I just create a branch with the version name with each release? Then merge the branch to main branch, push from main branch and then leave the version branch?

burnt ibex
merry patio
burnt ibex
#

Uh, not that I know of

#

But if you're doing it on every push/pull request

merry patio
burnt ibex
merry patio
#

I guess it won't in the long run

#

Any advice which action to use?

burnt ibex
#

I haven't used either of them myself, so I dunno

plucky root
#

Hey guys, I recently started learning python and wanted to do some projects. Can you guys tell me what are you guys working on or maybe suggest me some projects that I can do?

vague silo
plucky root
#

Thanks. I'll try my best.

dusty jewel
#

Does anyone here have experience with Cloudinary?

gilded lichen
#

hi! I am using code with me plugin in pycharm

#

i used to see files before

#

but now i can only see external libs

floral smelt
#

But it's pretty much handy for Python linting+formatting

#

You could try with Node modules too

#

Oh no wait, forget what I said. I misread and just saw pre-commit

#

Never used Github actions yet

#

In fact idk how I should get started with Github's CI

heavy knot
#

Is there a python bot development discord server?

burnt ibex
heavy knot
brisk lake
#

why on earth does this fail on the poetry installation

FROM python:3.7-alpine

COPY . .

RUN apk update && apk upgrade

RUN pip install -U poetry
RUN poetry install

RUN chmod +x /entrypoint.sh
ENTRYPOINT [ "/entrypoint.sh" ]
# 
#8 111.9     unable to execute 'gcc': No such file or directory
#8 111.9     error: command 'gcc' failed with exit status 1
leaden tartan
brisk lake
#

but it works just fine with the regular python images

#

why does python-alpine need extra c shit

leaden tartan
leaden tartan
brisk lake
#

python-alpine has fewer features than python images?

leaden tartan
#

Idk what you mean by features

#

But yeah it's less functional

brisk lake
#

thats sad

leaden tartan
#

You won't be able to connect to postgres db in a normal alpine image

brisk lake
#

ok so how do i figure out what im missing

leaden tartan
#

You would need to install more "c shit"

brisk lake
#

i need git and poetry for my project

leaden tartan
brisk lake
#

oh i see them

#

ok thanks

wooden ibex
brisk lake
#

Turns out i could use git from python:3.7 so imma just switch to that and drop whole minutes off the build process

#

Idk why i never checked if i could or not

#

Only thing i needed really was git and poetry

cinder wharf
#

hi all! I'm observing some memory issues in production but i'm unable to reproduce locally. Are there good tools / methods for doing forensics (taking / analyzing) on a heap dump for a uvicorn process? I'm able to divert some traffic away from an affected container to eliminate any impact involved with doing so

vague silo
#

hi @cinder wharf, this is actually an issue I was facing recently, I found tracemalloc to be the most helpful despite it needing a bunch of glue code. there's also pympler (https://github.com/pympler/pympler) but I haven't tried that out yet

#

my colleague loves pympler though

cinder wharf
#

@vague silo nice - I did find pympler but I was hoping there might be a tool that would work against a running process -- akin to how one would take a heap dump with jvm or ruby ... tho I guess i could maybe add an endpoint to my app that I can call to run pympler ?

vague silo
#

what I did in my app was enabled tracemalloc if an env variable was set and then save a tracemalloc snapshot on SIGUSR1

cinder wharf
#

ah nice - I may try that same method.. thank you!

stray tree
#

@vague silo Congo for helper

vague silo
warm coral
#

Hi there. I am looking for a tool that could help me use declarative syntax to define some layout. I found it in prompt-toolkit. But it is built in... i just need the generic idea or tool to do it. It would look like this...

from something import Root, Node, Leaf
xml = Root("Home", { "color": "red", "owner": "jack & jill" },
  [
    Node("Bedroom", { "color": "blue", "used_by": "jack" }, Leaf("Do not disturb")),
    Node("Kitchen", { "color": "yellow", "used_by": "jill" }, Leaf("Breakfast ready")),
  ],
)

To be precise, something like SwiftUI syntax.

stray tree
vague silo
#

aaaaa

#

thanks

peak nexus
#

can anyone help me with selenium , I am getting error while running it

vague silo
peak nexus
#

thannks, pls allow me .. it's quite long

vague silo
peak nexus
# 
from selenium import webdriver

driver = webdriver.Chrome(executable_path=r'C:\Users\viash\Downloads\chromedriver_win32.exe')
driver.get('https://www.google.co.in')
print("Page Title is : %s" %driver.title)``` this is the code.. and I will post the trace-back on the link which you provided
peak nexus
vague silo
#

hmm this is kind of weird, on one hand you give it an explicit path to the executable, on the other hand it complains that the executable needs to be in $PATH. just to be sure, that file exists and is executable?

#

if yes, then I would advise you to put it in path regardless of the executable_path you pass it

peak nexus
peak nexus
#

C:\Users\viash\Downloads\chromedriver_win32 this file path and I added .exe at the end

vague silo
#

and that is executable, right? did you try running it?

vague silo
peak nexus
#

yaa it's .exe

vague silo
# peak nexus

but this looks like the path would be C:\Users\viash\Downloads\chromedriver_win32/chromedriver.exe, or am I misunderstanding the explorer?

peak nexus
#

I am trying it now

#

I guess that worked

#

Now getting different error

#

if you free can you please check what's that is

vague silo
peak nexus
#

Thanks I will check 😇

vague silo
#

let me know if it works!

peak nexus
#

sorry, it's taking too long I have old PC so, it awlays hang

#

I just noticed it worked new tab have been opened

vague silo
#

no worries, take your time

peak nexus
# 
from selenium import webdriver

options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('--disable-infobars')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--no-sandbox')
options.add_argument('--remote-debugging-port=9222')

driver = webdriver.Chrome(executable_path=r'C:\Users\viash\Downloads\chromedriver_win32/chromedriver.exe')
driver.get('https://www.google.co.in')
#

This worked perfectly

peak nexus
vague silo
#

glad to hear 👍

peak nexus
#

Thank you so much @vague silo 😇 you are awesome

rapid sparrow
heavy knot
#

hello, what's a good comprehensive reference to learn this part of the devops roadmap.sh roadmap?

unknown.png
#

it looks like this will take some time

median storm
#

Hey, I am new to learning git/github, and am trying to push a git repository to github, but I got this error

# 
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.```
sinful siren
#

Check file permissions on your .ssh/ folder and keys

#

ssh -Tv git@github.com

clever coral
#

how do i host a image in digital ocean and use it in react

#

like in aws you host media file in s3 bucket what is equivalent for digital ocean

median storm
sinful siren
#

yeah you need to setup an ssh key if you don't and then cat the public key which ends in .pub and then go add it to github

#

if your keypair name is not a standardized name like id_rsa id_e25519 ... etc you may also need to specify the key name to use in an .ssh/config

#

nano ~/.ssh/config

Host github.com
    Hostname github.com
    IdentityFile ~/.ssh/some_key_rsa
    Port 22

Host gitlab.com
    Hostname gitlab.com
    Port 22
    IdentityFile ~/.ssh/some_key_rsa
#

the path in the identityfile is for the PRIVATE key path just to be clear it can be easy to do it backwards

median storm
sinful siren
#

yeah

#

then you can add it and it should work granted you do't customize the name of the key to nonstandard name

median storm
sinful siren
#

yeah its different path in windoze

#

one sec

#

might not exist either

median storm
#

Ah I figured windows might be different

#

I remember I downloaded ubuntu a while back and set up git and an ssh key and it worked fine-ish. But I wanted to try to get everything set up through windows so I wouldnt have to switch between cmd and wsl terminals constantly

sinful siren
#

with or without .

%USERPROFILE%/.ssh/

%USERPROFILE%/ssh/

#

can be done pretty easily still in windows

#

just do the part for the sshclient part

#

should install what you need

#

in powershell

#

Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0

#

even on windows 10 just needs the openssh client and possible server service at worst and if you do the ssh-agent service which i recommend it should cache your ssh private key pass during the login session so not needed to enter for every ssh session/connection

sinful siren
visual oxide
#

So TIL, tools-and-devops includes packaging and virtual environment tools

#

how do I express:

install_requires = ["kombu ~= 5.1"],
extras_require = {
    "sqs": "kombu[sqs] ~= 5.1",
}

In poetry?

Another Example in the wild https://github.com/dask/s3fs/pull/362/files#diff-60f61ab7a8d1910d86d9fda2261620314edcae5894d5aaa236b821c7256badd7R26

For full context, I'm trying to port celery from setuptools to poetry https://github.com/celery/celery/pull/6874/files

https://github.com/celery/celery/blob/master/requirements/extras/sqs.txt#L1 that's the current implementation but I'm not sure how to port to poetry

rancid schoonerBOT
#

requirements/extras/sqs.txt line 1

kombu[sqs]```
tawny temple
#

I suspect it is not possible

#

The mechanism for defining extras seems to rely on them being defined as optional dependencies first. Of course that's a problem in your case since you cannot duplicate the "kombu" dependency.

#

If the extra definition references something that is not defined in the dependencies, then it just "empties" it in the lock file

#

e.g. ```toml
[tool.poetry.extras]
sqs = ["kombu[sqs]"]

becomes 
```toml
[extras]
sqs = []

in the lock file. Not sure what exactly that means, but it doesn't seem to work.

#

Even if the extra isn't explicitly defined by me, the lock file seems to contain the dependencies for all extras of all dependencies. However, it doesn't seem to affect poetry install -E sqs

visual oxide
#

Hmm weird

#

It was only in pip with the 2020-resolver

median storm
# sinful siren Did you get it working?

Yeah sorry. I ended up finding some instructions on GitHub that set it up through git bash, and then it worked fine pushing on cmd. Thanks for pointing me in the right direction because I had really no clue ssh keys

sinful siren
median storm
visual oxide
tawny temple
#

I think they would have to introduce new syntax to solve this, something like ```toml
kombu = {version = "^5.1.0", extras = [{"sqs", optional = true}]}

#

That wouldn't be a breaking change.

#

Kind of clumsy though

visual oxide
#

I think having it not be a dict and instead be a toml v1 heterogeneous list of deps would help

#

I don't really understand the optional=true syntax

rancid schoonerBOT
#

setup.cfg line 96

all_non_platform =```
visual oxide
#

Why does replying to a question imply it's an answer?

tawny temple
#

Probably cause you asked under Q&A

mint tusk
#

Does anyone know why Vscode isn't showing any of the packages I installed with conda?

#

I've activated the conda environment I installed them in

#

But none of my imports work

#

I just realized, how would this python file know where my conda modules are?

tawny temple
#

Did you set the Python interpreter in VSCode to use conda?

mint tusk
#

Nope. How do I do that

#

Or I guess I can google that

tawny temple
#

ctrl shift p and then search for "python interpreter" or something

#

Then it will show a list of ones it detected, and you can select the conda one

mint tusk
#

Yep that worked. Thank you

teal mist
#

i keep getting a dir does not appear to be a git repository error when trying to push to a server, but im p sure the dir is a git repo, i checked and it has the .git

rugged flame
#

!warn 355040211702513666 Please get permission from the admins before promoting within the server. Contact @balmy mulch

rancid schoonerBOT
#

:incoming_envelope: :ok_hand: applied warning to @uneven flint.

vague silo
teal mist
#

i made it

#

nvm i got it to work

fresh cedar
#

Hey guys, I'm looking for some feedback on what other people use as environment setup's in production systems

#

is this the right channel ?

#

Currently we're having issues because the environment we deploy in differs from the environment we develop in. So easy enough, you create a virtual env that uses mimics the environment in both locations right

#

but then how do you also mock out the systemlibraries like openSSl problems etc

#

another problem is that we have software A that's developed by a group at work which has dependancies, but software B that uses software A also has it's own depandancies (which clash)

#

So I don't know ow to resolve them in a stucturized manner. Do I install software A in environment A and software B in environment B (even though software A needs environment A) or are environments only used when your packages/software are separated ?

jovial wing
#

Conda handles system libraries

#

Afaics, it seems like you either have to be able to guarantee the same system libraries on all machines, or use conda, or use docker

#

For your other issue, conda does have a sat solver which solves dependencies in a principled way, the ideal solution is to have a common environment meeting the dependencies of both A and B but obviously if dependencies are incompatible, then they're incompatible

#

If A and B are only run as two separate processed then you can also just have a separate environment for each and activate the correct environment just prior to launching

#

@fresh cedar

#

we do this anyway for our software, not that we have an A and B but we have versions of our software that keep getting deployed, and sometimes of course the dependencies change, which means the environment changes. And in principle we could be running different versions of our software on the same box. So it's just a given that there is no notion of a "global" environment on prod boxes. When the software is deployed, the deployment code ensures that the matching environment is available on that box. And then the software is executed via a bash wrapper that sources the correct environment just before launching. So you can have different versions with different environments coexisting on the same box and not caring.

fresh cedar
#

thanks, I guess what I was trying to say is that conda might help, but it doesnt resolve dependancies. Poetry can actually look at all packages you wanna use and try to mediate in upgrading/downgrading libraries so all software works together nicely

#

I'm having trouble to articulate the problem, but basically, we build a pipeline, which has package A that uses libA v1.1 and package B that uses libA v1.5 so they clash

#

I need to solve the dependancy when they are installed on the same system

#

so either I run package A in environment A (so it can use libA v1.1) and then use package B to run in environment B so it can use libA v1.5

#

if that's not possible (it should run in the same pipeline) then I should probably use poetry to mediate and come to a shared dependancy so it upgrades the first and downgrades the second (as i think poetry is quite good for)

#

So i think the questions boils down to 'can we use conda in the same pipeline to use packages in their own environment per step of the pipeline, or should we then sync the packages as a whole so the whole pipeline uses the same environment'

#

thanks for your answers so far @jovial wing

jovial wing
#

Np

#

@fresh cedar I have a decent amount of experience with conda, it's what we use for both C++ and python package management, so happy to answer questions related to conda in particular

#

I highly recommend conda, I've been really happy with it

pliant jungle
# 
  File "c:\Users\Jannik\Desktop\diablo3 bot\main.py", line 32, in <module>
    result = cv.matchTemplate(screen,smith,cv.TM_CCOEFF_NORMED)
cv2.error: OpenCV(4.5.3) C:\Users\runneradmin\AppData\Local\Temp\pip-req-build-sn_xpupm\opencv\modules\imgproc\src\templmatch.cpp:1164: error: (-215:Assertion failed) (depth == CV_8U || depth == CV_32F) && type == _templ.type() && _img.dims() <= 2 in function 'cv::matchTemplate'``` Hello guys i try to compare 2 images but i get this error. Does someone ever faced this? i cannnot figure it out
#

well, it was the file format, jpg works png didnt

floral laurel
#

Hello, I am trying to optimize my k means cluster code to wrk with an inputed amount of centroids can someone help me?

brazen obsidian
floral laurel
#

can u?

subtle kraken
#

My ADB Framework!

rancid comet
topaz aspen
#

anyone know what the max image size is for a docker image within gitlab CI ? I can't seem to find this info for some reason.

#

ping me if you respond please

#

Note - the image size i have is ~4.8 GB

barren iron
#

I don't think there is a max?

topaz aspen
#

this is in the build stage, so there's a max somewhere i assume

barren iron
#

Wow. I would question why your docker image is so big?

#

Can I also see the full error log?

topaz aspen
#

i don't see the relevance - assume it's that size because it's needed

barren iron
#

Okay... Fair enough

topaz aspen
barren iron
#

That's not really enough to go off for me to be much help anyways. Really sorry!

topaz aspen
#

it fails - on the error posted, looking at a few it seems to be during the pre-commit stage

#

but again, it's failing on this error, due to lack of space on device

barren iron
#

You said it's gitlab CI. Are you using this image as the base for CI?

#

Are you building it on gitlab CI?

topaz aspen
#

So one would need to know what the device is, how to determine it's space limits, and how to change those i presume 🤔

#

this is being built in gitlab yeah - so there's a .gitlab yaml file and a Dockerfile for it

barren iron
#

Right. Can you build it fine when it's not on gitlab?

topaz aspen
#

Yeah I've built it locally

#

that's how i know that it's ~ 4.8 GB from looking at docker images

barren iron
#

Right.

topaz aspen
#

(i'm not sure how i'd check that info in gitlab tbh)

barren iron
#

df -h checks space

#

Past this the other issues on Google all give more information, a full log of the error, the gitlab config. I'm aware your project is private, fair enough but it's hard to help with such a specific error

#

Are you using the default gitlab runner or your own?

wooden ibex
#

And yes, running your own runner will probably bypass this

barren iron
#

My point is... There must be a way for you to break up the image?

topaz aspen
barren iron
topaz aspen
#

I don't know how much i need to break it up by if i don't have any information on the size of the device that i'm working with i guess

wooden ibex
#

So if you need to build that large of an image, you will likely need your own runner.

wooden ibex
wooden ibex
topaz aspen
#

doesn't say anything about how much space there is tho

rancid schoonerBOT
topaz aspen
#

I'd managed to parse that "no space" meant there was no space

barren iron
#

The runner is a VM. This means it's space is not a finite number 24/7

wooden ibex
barren iron
#

It changes. There's no way to give you a value

wooden ibex
#

if you are paying customer, you can open a ticket and ask and likely get a firm response

barren iron
#

And gitlab don't give out the public specs of their VM's storage

topaz aspen
#

I am a paying customer yeah - but if i have to try and reduce the image size then it's tricky without knowing what to aim for.

It changes. There's no way to give you a value

ah, ok - this explains why repeatedly running has led to some of them building

wooden ibex
#

4.8GB build image is pretty unusual

barren iron
#

If you are a paid customer go directly to gitlab

#

They can probably suggest a quick fix

topaz aspen
#

not saying that makes it usual

#

looking at du from / i have:

4.0K    ./srv
45M    ./var
4.0K    ./boot
2.4G    ./opt    * 
4.0K    ./mnt
0    ./sys
1.6M    ./tmp
2.7M    ./etc
20K    ./run
0    ./dev
1.4G    ./root   * 
1.2G    ./usr
4.0K    ./media
4.0K    ./home
wooden ibex
#

4.8GB of packages is still a ton

#

I wouldn't trust df 100%

topaz aspen
#

*boost

#

granted, that is the largest

tawdry needle
#

1-2 GB conda environments aren't unusual

#

4-5 seems very high

velvet spire
#

istg why is gpg so hard to use

deep estuary
#

gpg is neato

velvet spire
#

it used to work on this device and it just...stopped

deep estuary
#

hahah

thorn sentinel
#

Hi all, a question about pyenv: I am trying to set up a project with a different python version than my system one. The problem is that when I try which python in the cli in the project folder, it points to the system default irrespective of where I am in the tree (including $HOME directory). pyenv which python points to the version installed by pyenv, again, irrespective of which folder on my system I am in. What am I missing?

deep estuary
#

you need to add the pyenv init to your shell startup script

#

try run pyenv init @thorn sentinel and can you share the output here

thorn sentinel
#

@deep estuary you mean eval "$(pyenv init -)"

#

?

deep estuary
#

aha, that is in your shell?

thorn sentinel
#

I am adding it to .bashrc now, see where it takes me 🙂

rancid schoonerBOT
#

Hey @thorn sentinel!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

thorn sentinel
# 
# shell setups. See the README for more guidance
# if they don't apply and/or don't work for you.)

# Add pyenv executable to PATH and
# enable shims by adding the following
# to ~/.profile:

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"

# If your ~/.profile sources ~/.bashrc,
# the lines need to be inserted before the part
# that does that. See the README for another option.

# If you have ~/.bash_profile, make sure that it
# also executes the above lines -- e.g. by
# copying them there or by sourcing ~/.profile

# Load pyenv into the shell by adding
# the following to ~/.bashrc:

eval "$(pyenv init -)"

# Make sure to restart your entire logon session
# for changes to profile files to take effect.```
#

The thing is i have the following in the following files

#

.bashrc:

eval “$(pyenv init -)”
eval “$(pyenv virtualenv-init -)”

.bash_profile:

export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"
velvet spire
barren iron
thorn sentinel
#

@barren iron read what sorry?

barren iron
#

Click what I replied to

deep estuary
#

I've not seen virtualenv-init before

barren iron
#

The message from the python bot

deep estuary
#

in fact it's not a command

#

huh

thorn sentinel
#

@barren iron this looks like a message about why it's not a good idea to post txt files

barren iron
#

It also mentions code-blocks... If you read down to the bottom

thorn sentinel
#

@barren iron gotcha - thanks