tools-and-devops

[crude summit]

shared the examples

[rough marlin]

using a regex tester / simulator site might be helpful

[floral smelt]

Yeah like regex101

But since these addresses are separated by commas, a split(",") can do a first work. If you want to exclude Address: then you can use ^Address: (.+), then use re.search(pattern, string) to get a match, then extract the first group (between parentheses) with re.search(pattern, string).group(1)

[crude summit]

These addresses are part of patient details shared in tweets

the problem is, there might be some addresses where it's not explicitly mentioned it's an address

people can just share the address and ask for help

is there any way to use regex for that problem?

It's a real big problem and the solution is gonna be used in real time

[floral smelt]

If you have a clear pattern for address, you can. It's more complicated if the address in question is only words separated by 2 or 3 commas.

A good example of address would be Street Number - Street Name - City - State - Postal code - Country

[crude summit]

the problem with that is, people don't use that format when in emergency.

they just hope that a human reads the address and try to communicate with them

but those tweets get lost

[rough marlin]

I don't know if you'll be able to apply regex to this successfully

If there's no standard format used, then regex isn't as powerful

it might be able to help you catch groups delimited by newlines and :

[crude summit]

Is there any way to combine regex with google maps?

[rough marlin]

well

yes, but that assumes a consistent format

regex is good for looking for structured patterns in text

[floral smelt]

Best chance is to compile every single address and use a search system. Probably Elasticsearch.

Or train a ML model for address suggestions

[rough marlin]

yeah if there's a source you can use to pull from like an open streetmap database dump for india, that would be very helpful

imo, don't try to pull directly from their API on this unless they ok it ahead of time

reaching out to them for advice might be helpful too

[shy locust]

I see enough of a pattern, that you might get most of them with several regex parses

[rough marlin]

regex seems to be best for matching and extracting data from well-structured non-recursive

if you can get human volunteers to help, you might be able to set up a system that marks poor matches for further review by the humans

[crude summit]

I'm thinking to compile list of cities for now, and see if i can use it in regex

[shy locust]

Is this a case where anything is better than nothing?

[crude summit]

starts with Location|Address and ends with City Name

[crude summit]

Well, I gotta do something.

[crude summit]

That sounds good.

[rough marlin]

the good thing is that if you get this to take off and get humans to help extract data and to mark the quality of machine-created extractions, you can also start building a data set someone else can use to build ML systems to further tune the use of human assistance to be as effective as possible

[crude summit]

that can be done

[rough marlin]

focus on one thing at a time imo

[crude summit]

but I never worked with ML before so idk how it can be done

I agree

[rough marlin]

then focus on getting the MVP working

and get help from someone with experience

[crude summit]

https://github.com/gagangulyani/COVID-Text-Extractor

I wanna implement it in this

if someone can contribute in it, it will be really helpful.

I'm gonna add PDF parser today

and see if I can make a telegram bot using it

It's poorly structures, I know, but I'm so new to this and we need to find a working solution asap

[rough marlin]

find a working solution asap
is there a specific problem you're trying to solve?

[shy locust]

I can try and help, over the weekend when I'm off work

[rough marlin]

this looks like a general text / image extraction tool

[crude summit]

yes, tons of tegeram messages have numbers in images

[rough marlin]

but i'm not sure i understand what the usecases

[crude summit]

ocr is needed

[rough marlin]

so are people sending photos of paper with text written and printed on them?

imo, humans will do better than machines at this if it's OCR tasks

[shy locust]

Doubly so if handwritten

[crude summit]

you can go through the test directory in my repo

[rough marlin]

but the problem is also organizing the humans

[crude summit]

you will find most of the images which are real

1 is dummy for testing ocr

[crude summit]

that will be great! 🙏🏻

[rough marlin]

I think a better statement of what the problem cases to be solved are

[crude summit]

I'm figuring it out as well.

[rough marlin]

but I might be interested in helping

[shy locust]

Are you parsing simple chat channel text or photos/images tho, chat channel text is one thing

[rough marlin]

my hardware is very underpowered for running ML or CV tasks, but there might be ways to sidestep that

also

i'm curious

[crude summit]

both

[rough marlin]

why is python 3.9.4 a requirement?

[crude summit]

I haven't made a bot yet

but it's intended to work for images and pdfs

ocr is poorly done so regex was needed to extract what i could

[supple venture]

might be better to move to a help channel, since this is not really on "tools and devops" rn

[rough marlin]

hmm

[crude summit]

heading to #help-cake

[ocean spear]

yo guys anyone know how to learn python modules specifically the chess one?

[sly sleet]

which chess one

[supple venture]

by checking there documentation

you can't find tutorial for every single library

[nocturne gazelle]

HI, I made a new workflow package: https://github.com/flowsaber/flowsaber any issues and advice are appreciated.

[rapid sparrow]

def attach_root(bot, storage, chanell_controller) -> commands.Bot:
    "attaching root commands to application"

    @bot.event
    async def on_ready():
    "doc string which I try to extract"
        print('We have logged in as {0.user}'.format(bot))

    return bot

is it possible through autodoc of sphinx to acceess somehow docstring of a function declared inside a function?

[heavy knot]

i hate decorators

what if you return on_ready instead/

[agile lark]

Is there a general agreed-upon convention for Git commit message body line length?

The second part, for clarity. I have seen longer line length in larger open-source projects like but I feel like long lines is gonna royally screw over some people

I have found out that the convention is 50/72 (50 for first line, then space, then generally 72 char/line). I'll just follow that rule

[nocturne gazelle]

It's possible, you need to modify the generated class using a meta class or decorator, the main idea is that, the class's docstirng and class's method are similar things(attributes) that can be modified after

[supple venture]

yes thats what is generally recommended

you can checkout this resource https://chris.beams.io/posts/git-commit/

GitHub displays up to 72 on the summary line, so I use that.

Another thing i like to include in tests, inspired by mark, is to the main feature you are writing tests, ik that the summary line is really short, but adding the main feature at the start, is helpful when you scroll through the commits

for example, say you writing a tic tac toe game

i would do it like this: (tic-tac-toe): ...

[tawdry needle]

It helps in a monorepo too, for example:

* 3cb0fd (master) auth: fix xsrf check
* 7e9b4e homepage: adjust button placement
| * 1a0925 (feature/db/sql-haversine): db: haversine dist in pl/sql
|/
* 57f6c4 homepage: wcag compliance in footer

where the homepage: part says what part of the repo is affected

some people even use emoji to represent the type of commit (bugfix, feature, etc) which saves some characters i guess, but i don't place too much value on distinguishing between bugfix, feature, etc. commits. because commits should be small!

[gilded lichen]

Hi, as you may know Tika needs Java installed as it adds server.jar in the file directory. But I want users to be able to use my program without Java or Python(will convert it to Exe)
is there a way to add server.jar to the temp folder of the user

[tawdry needle]

normally on windows i think you add files like this to the installation location (e.g. C:\Program Files\Frost14 Amazing Program\server.jar)

[lethal stump]

How do I set python3 version to 3.9.5

pyenv global 3.9.5 don't work

And I did use pyenv install 3.9.5

[gilded lichen]

@tawdry needle wow that's helpful

but how do i add files

making an exe btw

[tawdry needle]

@gilded lichen i think you would want to create a windows installer that puts the files in the right place, adds the program to the "Add/Remove Programs" thing in control panel, etc. not sure how to do that, though.

[tawdry needle]

be specific. what does "don't work" mean?

[lethal stump]

Simply when I use python3, it brings up 3.8 and not 3.9

[gilded lichen]

can you be more specific?

maybe a tutorial on yt

[lethal stump]

I need my dpy bot to run on 3.9, or it will error

[tawdry needle]

is pyenv configured correctly in your shell startup scripts? it sounds like maybe python3 is referring to your system-installed python, not the pyenv version.

[tawdry needle]

i don't know how to do it. but you will want to look up how to create a windows installer

[lethal stump]

Fixed it by using update-alternatives

[tawdry needle]

i don't think it's good to use update-alternatives for pyenv, but if your distro has python 3.9 in their repo is guess that's fine. however please do not use sudo pip install - make a virtualenv

you really should be using pyenv

[supple venture]

i don't really like emojis, for some reason, i like to keep it text

[tawdry needle]

i dont like it either

[lethal stump]

When it's not working...

[wooden ibex]

Toss it into a container

[supple venture]

What env are u using?

[sly sleet]

@lethal stump can't you use a venv for that?

just create the venv w/ python3.9

[last plume]

hi

[lethal stump]

OS?

[supple venture]

No, like pipenv, poetry, venv

[lethal stump]

tbh I'm not certain

[supple venture]

ah, how do you run your code now?

[lethal stump]

I use 3.9 interpreter
My IDE is VS Code

Well on the VPS, I simply type

python3 /usr/Reactaio/Reactaio.py```

[supple venture]

show the output of python3 --version

[lethal stump]

arshia@D7k6Wib5:~$ python3 --version
Python 3.9.0+```

[supple venture]

hmm, that should work then

[lethal stump]

Yeah

Hopefully

[supple venture]

it does use python3.9

[lethal stump]

I used update-alternatives

So that's why

[supple venture]

ah i see

[heavy knot]

why

[lethal stump]

cause people

[quick cave]

What would be a good tool to use to fill out a PDF (or generate a PDF) from a data frame? Some of the libraries that's I've found seem to be outdate.

[fallen shoal]

Just a quick question: I want to have a venv with the latest version of Python 3.9 but my Anaconda install keeps Python at version 3.8? From what I've read around there is no way of having a venv that is a later version than the base environment. Is there any way of getting around this - keep my conda install unaffected and have the latest version of python?

[tawny temple]

Yes, you could install Python separately outside of conda. You'd probably need to specify the full path to the binary when you want to use it, but you could get around that by creating a symlink to it with a different name (if it does not already do that for you)

You could also just install Python 3.9 with conda by creating a new environment

[fallen shoal]

I don't seem to have that option. It maxes out at 3.8

[tawny temple]

I've never bothered using the UI. I could tell you how to do it with console commands, but my guess is that your conda is just out of date.

If you're more comfortable with the UI, then try to find how to update conda through the UI (I don't know since I don't use the UI). I could tell you the console commands for it though.

[fallen shoal]

so yesterday I ran conda update --all twice and it still held at 3.8

I am seeing this currently

I am comfortable with cmd

[tawny temple]

Try conda update conda then conda update anaconda

Or you could just try going straight to python 3.9 with conda create -n python39 python=3.9

The name can be whatever you want, of course

[fallen shoal]

but will be able to create a virtual environment with a version of Python I do not have? i will try it of course but just asking for my understanding.

[fallen shoal]

I am doing this at the moment

[tawny temple]

Yeah it should just pull it from the internet and install it

[fallen shoal]

and is conda virtual environment different from venv method? I've been taught to use venv for my virtual environments

[tawny temple]

It's pretty similar. I like to think of it as a semantic difference

Conda environments are like "base" environments.

If you want, you can use them by activating one and then creating a venv too.

But I suppose you technically can use conda to directly manage all your environments

I don't use conda that much anymore, so I can't comment on what is considered better. You may find it more convenient to just use a single tool, conda, rather than juggling around multiple tools.

[fallen shoal]

I see. I presume thought that conda virtual environment will pull all the anaconda packages into it whereas venv tends to be a very blank environment

[tawny temple]

No, I believe it creates an empty environment too, and you have to explicitly install anaconda if you want all those packages in it.

Like, by activating the environment and then conda install anaconda

Though that's not how venvs are typically used - you install only what the project requires.

If you're installing all of anaconda, that has the implication that you intend to share this environment with other projects as a "general purpose" environment, which can have its drawbacks if projects rely on different versions of packages.

[fallen shoal]

I am starting to understand the differences as at work some of our automation is done in clean python installs with frozen package versions. Whereas in my free time I primarily use python for data science and so don't really care about packages and all that

[fallen shoal]

I used this. It worked perfectly. Exactly what I needed. Thanks!

[tawny temple]

You're welcome

[tawdry needle]

Has anyone got hypercorn --reload working inside a docker container? It doesn't seem to reload anything when I modify a file mounted as a volume... I asked a SO question with more detail in case someone thinks they're able to help. https://stackoverflow.com/questions/67506424/hypercorn-with-reload-and-docker-volumes

[supple venture]

I think quackstack does that, one second

ah nope

[tawdry needle]

quackstack?

[supple venture]

https://github.com/python-discord/quackstack

[tawdry needle]

that does seem like the same setup as me, at least

[supple venture]

i will try to squash in a reload into that and test

You got an answer, hmm

building image...

@tawdry needle yep works for me

[tawdry needle]

weird, with my dockerfile? or with quackstack?

[supple venture]

did with quackstack

Dockerfile

[tawdry needle]

and if you modify the file, it reloads?

running it isnt the problem, it's that hot reloading isnt working. it just doesn't reload.

and i did confirm that the file change is reflected inside the container

[supple venture]

should i just modify anything in the main.py?

[tawdry needle]

yeah i think so?

[supple venture]

nope, doesn't reload

does it relaod withotu docker @tawdry needle ?

ok yeah it does

lemme give it another try, one min

[tawdry needle]

Might be related to the filesystem driver

Maybe inotify or whatever hypercorn uses doesn't work with overlayfs

[supple venture]

hmmm, would uvicorn work?

or it uses the same thing

[tawdry needle]

maybe, i could try it

[supple venture]

Yeah I get the same problem as you described on the stackoverflow link, I will get it a try with uvicron once get back to my pc

[inner pollen]

https://github.com/Agent-Hellboy/YouTubeDataApi_Wrapper

Does anyone help me in improving this library?

[tawdry needle]

it works with uvicorn

seems like a hypercorn bug possibly

[supple venture]

Cool thanks, I will see why this happens

[copper sparrow]

Hey!
I’ve my backend on django where user’s sign up via email.

I want to implement a email verification system for the user.
How do I go about doing this? Do I’ve to use a third party email validation provider? Or is there a way in house

[supple venture]

Not the right channel btw, should be meant for #web-development

But this is what I found on googling: https://stackoverflow.com/a/56116529

[copper sparrow]

Thank you!

[open granite]

Hey guys, I need to use a currency conversion py package/api (FOSS only) in my app. I need historic data (at date level granularity, at least). Should I use an api or go with a package? How do I compare the ones available? Any recommendations?

[hushed jungle]

Anyone have any experience deploying docker compose in aws?

[tawdry needle]

tfw you spend hours debugging only to realize there's a bug in docker and you need to update to a newer version

[rapid sparrow]

That is why I would need one day to spend some time in order to put docker building + launch of tests inside a pipline to be tested too

Had same issue, yesterday

[quaint zinc]

Not sure if this counts as as a tool, but i want to create an MSI installer (or sometype of universal setup installer thing) for my python script. I want it to have a license key so i am able to distribute it as paid. Anyone know of a good way to do this?

[rapid sparrow]

What would be the most easiest to learn quickly tool from zero for the task

I have multiple VPSes, with SSH accesses
I need to deploy to all of them, the same docker container but with 4 different environment keys to each of them?
preferably wishing to receive monitoring update of their alive status (but that's not necessary)

at the moment I use some machine to build container, and just push container to those machines / and relaunch in each of them with shell scripts(edited)

[shy locust]

To anyone experience in boto3
when you use get_paginator('action'), does paginator immediately perform the operation and store the results, or only when you call paginator.paginate?

For example, if I put a filter in the paginate call, will it only request that?

Does get_paginator or paginator.paginate make the api call?

[rapid sparrow]

you know... actually the most easiest way probably would be modifying slightly my script just in order to see alive statuses

and perhaps making it running with multi threading once it finishes uploading container

it should speed it up greatly and fit my current needs

[rough marlin]

Is there a preferred way to handle setting up a daemon / task scheduling in 2021? I've seen python-daemon and daemon , and I don't want to use cron because I want to make something portable.

[rapid sparrow]

redis/celery

if you want portability, use docker compose

if you want to be something not requiring docker compose that can fit inside your application without additional troubles

...well, you can just daemonize thread from multithreading

but celery is much more.... advanced solution

because it can give you level of control similar to cron, but at your application code level

in theory... there are ways to use it, while using as db for this your main database (sqlite/postgres).... but using Redis is more preferable for this

[wise hollow]

ay

[heavy knot]

Would you guys happen to also know how I could log my errors from journalctl? I'm so sick of doing journalctl -u [service name]

[tawdry needle]

write a script 🙃

[heavy knot]

I'm not sure how I'd do that, or where to even start

[tawdry needle]

@heavy knot do you use bash as your shell? if so, you can write something like

svlog() {
  local svc="$1"
  shift
  journalctl -u "$svc" "$@"
}

then you can write svlog -xe foo-service instead of journalctl -u foo-service -xe

[rough marlin]

i thought redis was a key/value storage system but ty, i'll look into celery

[rapid sparrow]

oh, I wrote ambigiously then, Redis and Celery (not or)

they are both used together for this task

[supple venture]

could you rephrase your question, a bit difficult to understand it, do you mean how to add untracked files?

[heavy knot]

https://slambook-py.herokuapp.com/nuc2vir
python web app made using django
give a try guyssss

[supple venture]

could show an example, screenshot?

yeah, it is your github username and password

nothing wrong in that

the error is?

show the exact error

umm

Update git version

git update

wait no, one sec lemme confirm

https://stackoverflow.com/questions/13790592/how-to-upgrade-git-on-windows-to-the-latest-version

yeah

not sure, whats the config on windows, just select everything

choose the 2nd one

which is recommended

ummm, dunno, lemme google

https://stackoverflow.com/questions/62456484/whats-the-difference-between-openssl-and-the-native-windows-secure-channel-libr

[supple venture]

umm looks weird, lemme have a look

[supple venture]

looks like git files got corrupted, not sure tho, try recloning the project

[brisk lake]

What are you trying to do with the repo anyway

[supple venture]

nah, just recloning your repo

[heavy knot]

are there any libraries which can implement OCR on a screen that is being scrolled live?

like for example if i want to make a program which constantly watches my screen and keeps converting text on the screen to like a dictionary and if a certain word pops up on the screen, it needs tro detect it and then shut the program off

its kinda like u have a screen recorder(like obs) and u are implementing ocr at the same time

[rapid sparrow]

FROM python:3.8-alpine as builder

WORKDIR /usr/src/app

ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

RUN apk update && apk add gcc python3-dev musl-dev  

# this line changes
COPY ./requirements.txt ./

RUN pip wheel --no-cache-dir --wheel-dir /usr/src/app/wheels -r requirements.txt

# === FINAL IMAGE ===

FROM python:3.8-alpine

RUN addgroup -S app && adduser -S app -G app

# Create directories app_home and static directories
ENV HOME=/home/app
ENV APP_HOME=/home/app/web
RUN mkdir $APP_HOME
WORKDIR $APP_HOME

# Copy dependencies from builder image
RUN apk update && apk add --no-cache libpq 

COPY --from=builder /usr/src/app/wheels /wheels
COPY --from=builder /usr/src/app/requirements.txt .

RUN pip install --no-cache --no-deps /wheels/*

COPY . $APP_HOME

RUN chown -R app:app $APP_HOME

USER app

RUN python manage.py collectstatic --noinput

CMD python manage.py runserver 0.0.0.0:8000

What is exactly advantage to using additional builder image + building through wheel?

additional building image saves space of image?
wheel makes sure to compile binaries, even if they aren't present for the current OS among prebuilt ones in pip?

what confuses me the most, what the hell gcc is doing there%

lol. properly made .dockerignore reduced size of my image from 1gb to 500 mb
and alpine usage instead of my old unoptimized ubuntu additionaly decreased to 200mb

that's I like

hmm proper dockerfile also heavily reduced rebuild times

[wooden ibex]

Did you try Python Slim?

But if you got Alpine working without tearing your hair out, congrats to you.

[rapid sparrow]

nice to know as alternative

I usually worked with just Ubuntu 20.04 before 😉
although used python:latest for Github/Gitlab actions usually

but yeah, thanks to Imagine guys, I have now working Alpine dockerfile

[wooden ibex]

Cool

[rapid sparrow]

their recipe is cool

much faster

due to requirements being separated into step before COPY main code

optimized to super level, size is nice too

[wooden ibex]

Yea, I never futz with alpine or multistage, it’s rarely worth it with python

[rapid sparrow]

shrugs, it looks like one time thing to learn

after that I can use it for any Python project as much as I wish

[wooden ibex]

Until you run into wheel problem because musl doesn’t work

[rapid sparrow]

uh. all right.

it works, for now)

[wooden ibex]

Some libraries lose their mind in alpine

https://pythonspeed.com/articles/alpine-docker-python/

[rapid sparrow]

nothing prevents me to use at least half of those optimizations in regular Ubuntu

it will be a step forward, I hope

[rapid sparrow]

well, until it happens I can use Alpine at least

after that I'll go to Python slim probably, or return to Ubuntu ;b

hmm or better to do that immediately

[wooden ibex]

If you want to stick with alpine, feel free

Just pointing out it can have weird issues at time

[rapid sparrow]

Scarying issues

[rapid sparrow]

built with python:3.8-slim, difference from this alpine scenario above, literally 10mb in size bigger

but shorter instruction... much shorter instruction)

thanks. good advice

it looks like much more stable long term solution

this alpine scenario still helped me though

separated from COPY . . requirements installations make building much much faster 😉

[rapid sparrow]

at the bottom of the article is mentioned

that this can be resolved

and it actually was accepted by python society a month ago
to include their support for musl

https://www.python.org/dev/peps/pep-0656/
https://discuss.python.org/t/pep-656-platform-tag-for-linux-distributions-using-musl/7165/32
Python.org

not sure which alpine version would include the fix for that though

https://github.com/psf/fundable-packaging-improvements/issues/26

it is surely still in development

[wooden ibex]

Work is ongoing, it will take a while and it’s possible that some packages will never get musl support

Personally I’m like, size of uttermost important yet let’s use python!

One of these things is not like the other.

[worn light]

Not sure if this is the right place but.... I am trying to run MSVC developer command prompt from python. Anyone done this successfully?

I am making a tool to setup a C++ development environment for Godot. It automates pulling some git repositories and should also build them

[brave knoll]

hope this is the right place to ask, I'm working with pandas in VS Code, anyone have any favourite extensions they'd love to share with me?

[night quest]

#editors-ides

It's just a suggestion

[median reef]

python' is not recognized as an internal or external command, operable program or batch file jenkins

its workin on windows command prompt but not here

[supple venture]

does restarting jenkins help? like reconnecting

[median reef]

np

[median reef]

no

[supple venture]

hmm, not sure but is python in jenkins path?

like adding python to path through jenkins, if that's possible

[median reef]

idk exactly how its done

not enought tutorials on net

[median reef]

i tried adding and environment variable ,failed

[supple venture]

not sure

[cloud prawn]

What are some good tools for brainstorming?
I usually do brainstorming on paper and whiteboard. I cant use paper because I need more space. And the whiteboard is already full with current ideas.

[strange basin]

virtual whiteboard

[supple venture]

Whiteboard is what I use haha, get another one or use online ones

[dense cliff]

Hello guys i had a small doubt
If we add a telegram add
Can we revoke its permissions after the use

[supple venture]

https://about.gitlab.com/blog/2021/05/17/prevent-crypto-mining-abuse/

To discourage and reduce abuse [of free pipeline minutes available on GitLab.com and on other CI/CD providers to mine cryptocurrencies], starting May 17, 2021, GitLab will require new free users to provide a valid credit or debit card number in order to use shared runners on GitLab.com. A user will be able to run pipelines without providing a credit or debit card if they use their own runner and disable shared runners. Although imperfect, we believe this solution will reduce the abuse.
.
When you provide the card, it will not be charged but instead will be verified with a one-dollar authorization transaction. No charge will be made and no money will transfer. A credit or debit card is one (of many) controls we have put in place to reduce abuse of our platform. We will never fully solve platform abuse, but the more barriers we put up, the more difficult and expensive it becomes to engage in abuse.

[rare ridge]

is there a guide on how to use a docker container and a cloud provider to set up a basic infrastructure

[rapid sparrow]

the great and powerful Flask Mega Tutorial covers deployment in linux servers with docker usage
https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvii-deployment-on-linux
https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xix-deployment-on-docker-containers

[rare ridge]

What linux distro should I use

[rapid sparrow]

I would recommend using Ubuntu 20.04, it is quite easy to manage
as alternative Debian 10 is good too

recommending to be never using Centos7 if it is possible

[rapid sparrow]

but, if you are going for Docker Container, it will matter little which Server OS you are using for deployment

because they make that abstract/isolated? and easy to launch in any OS

it becomes a matter to learn...

how to write good dockerfile script, it uses its own OS inside of it

better to get a hang of Ubuntu or Debian, because those are two usual best choices to use inside the container

[rare ridge]

ok

[rare ridge]

should I do docker container or linux server

[rapid sparrow]

those are two things which are better be used together.
You can deploy docker without linux server to some services
As well as you can deploy into linux server without usage of docker

But usually you get Linux server, install docker service there, and deploy your container into it

[rare ridge]

is there a guide on how to install and setup docker or is it a pretty easy thing to do?

[rapid sparrow]

official docker guides are the best in my opinion
here example for Ubuntu
https://docs.docker.com/engine/install/ubuntu/

usually I would recommend also adding your user to docker rights after that though
https://docs.docker.com/engine/install/linux-postinstall/

[rare ridge]

I have a window system unfortunately

[rapid sparrow]

which windows

[rare ridge]

10

[rapid sparrow]

windows 10 supports docker now as far as I know

I did not try it though, not sure how much pain in the ass it is

[rapid sparrow]

https://docs.docker.com/docker-for-windows/install/

looks like there is official instruction

I would honestly recommend better renting any Linux test server and working with docker there probably, instead of Windows usage

(as well as migrating to Linux for development once you are ready)

it is just much easier to learn/make better when you are using Linux to have everything ready for deployment to Linux server

there are a lot of additional services, back end / dev ops needs when dealing with Docker

I think it would be nearly next to impossible to use them in Windows

[rare ridge]

could I use a google cloud provider to use a Linux server

[rapid sparrow]

!provider

?tag largehosting

Free:

• Openshift: https://www.openshift.com/
  2gb of memory, automatically expires after 60 days, but you can resubscribe as often as you'd like.
• Heroku: https://heroku.com/
  No CLI or SSH, no file access, might make your app sleep if you reach monthly uptime limit.
• AWS (Free tier) https://aws.amazon.com/free/
  Free tier containers from Amazon, expires in 12 months
• Google Cloud Platform (Free tier) https://cloud.google.com/
  Free tier google cloud servers

Paid:

• Azure https://azure.microsoft.com/en-us/
  Starting at $1.69/month for their B1LS virtual machines, 0.5GB of ram and 4GiB of SSD storage.
• OVH: https://www.ovh.com/us/vps/
  Full VPS starting at $3.49USD/month, choice of OS, high reliability.
• Digital Ocean: https://m.do.co/
  Starting at $5/month (USD), you can have your own server with 20GB SSD Disk, and 512MB Memory. Use https://bithost.io/ if you want to pay with cryptocurrencies.
• Linode: https://www.linode.com/
  Starting at $5/month (USD), you can have a server with 20GB SSD Disk, and 1GB memory
• Vultr: https://www.vultr.com/
  Starting at $2.5/month (USD), you can have a server with 20GB SSD Disk, and 512MB Memory
• Amazon(AWS) Lightsail: https://amazonlightsail.com/
  Starting at $5/month (USD) (first month free), you can have your own server with 20GB SSD Disk, and 512MB Memory.
• VIRMACH: http://virmach.com/
  Full Windows and Linux Desktop VPS starting at $7USD/month and $10USD/month respectively.
• Sloppy: https://sloppy.io/
  Starting at 5$/month (USD) You can have 500MB Memory, 1TB Transfer per month and 16GB of storage. Extra
  storage and ram can be bought.
• Galaxygate https://www.galaxygate.net/
  Starting at 3$/month (USD), get 15GB of storage, 1GB of ram, and unmetered bandwidth.
• Hetzner https://www.hetzner.com/cloud
  Starting at 2.49€/month (EUR) for 20GB of storage, 2GB of ram and 20TB of traffic.

[rapid sparrow]

here you go

shortly speaking yes

[winter spruce]

can someone please help with increasing ulimit on windows ubuntu

[tawdry needle]

that is a heck of a list, thanks for sharing

[rapid sparrow]

taken from another major programming Discord server, if allowed can say its name

[toxic bough]

you can say its name, it's fine 😄

[rapid sparrow]

all righty. taken from here
https://discord.gg/code

[toxic bough]

cool

[median reef]

my java program works on ubuntu terminal but not on jenkins

[night quest]

This is Python server

[rare ridge]

for openshift do I pick developer sandbox as a plan

[wooden ibex]

So much you were told was wrong and I’m mobile I can’t type out correction but you don’t need that for development.

But docker runs fine on Windows with Linux containers and it’s pretty much same experience you would get on Linux with rare exceptions related to hardware emulation which few people care about.

[wooden ibex]

There is official instruction because it's been supported for at least 2 years and for past 8 months, been supported under WSL2. There is nothing Linux adds to development when dealing with Docker and everything you can do on Linux with Docker, you can do on Windows, support is completely there. Recommending Linux is showing clear ignorance of Docker Ecosystem on Windows.

[rare ridge]

I don't think he is ignorant tbh I think he is used to using Linux with dealing with development, and doesn't some Linux distros come with tools that help with development.also what did he tell me wrong

[wooden ibex]

Exactly what I said, using Linux as OS for docker in development environment isn't required therefore the recommendation ```I would honestly recommend better renting any Linux test server and working with docker there probably, instead of Windows usage
(as well as migrating to Linux for development once you are ready)

[rare ridge]

But he recommends me to go to Linux and never mentioned that it only worked on Linux heck he even linked on how to install docker on windows

[rare ridge]

and he says to migrate to Linux for development once I am ready

[grand anvil]

@wooden ibex ping

how can i run python scripts for ever ?

on linux vps

(flask project)

anyone?

[heavy knot]

can any suggest me tool or someting so i can change whole python code to power shell script so that script can run in pc that have not python installed??

[wooden ibex]

There is no easy way to do that, you will have to rewrite into Powershell

[heavy knot]

how can i ?

please tell

?

[wooden ibex]

There is no reason to migrate to Linux for Development unless you are doing something that requires Linux as base OS (like kernel or hardware development) Windows == Linux for most development work and sometimes Windows is easier esp if you insist on not running mainstream Linux distro like Ubuntu/RHEL/Debian

That is my point, the notion that Linux is superior to Windows for development is false in most cases.

[heavy knot]

can you plaese tell?

[wooden ibex]

Open up VSCode, write Powershell?

[heavy knot]

ya how to convert python file to powershell script

[wooden ibex]

You can not, you will have to write the Powershell yourself

[rapid sparrow]

I am lazy to argue

my argument is simple

Our websites go being deployed to linux servers

within devops grasp of skills, it is easier to learn everything, how to deploy for Linux server, while you are having Linux desktop

[tawdry needle]

That doesn't make Linux a better platform for development, it makes Linux a better platform for learning about Linux.

[rapid sparrow]

When I switched to Linux, I feel like my whole OS became one big IDE

any tool can be easy to install

[winged frigate]

Distros like Pop!_OS comes preinstalled with most of the tools you need to get started with development

[rapid sparrow]

i prefer clean Ubuntu 20.04 LTS

with it being LTS and one of more or less late versions, it has everything I need to install from snap
or having good guides as alternative

[supple venture]

you can create a simple systemd service file for it if you are running Ubuntu.

[Unit]
Description=A test unit

[Service]
ExecStart=<full_path_to_script>
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=test

Put these lines in a file called test.service in etc/systemd/system. Now reload systemd and start the service.

An alternate option is to use nohup

[wooden ibex]

You install WSL2 and get pretty much same experience

[winged frigate]

I like Pop for it's Nvidia support, but Ubuntu is nice too

[wooden ibex]

and keep Windows if you have a need for it

[rapid sparrow]

I installed windows to Virtual Machine

in case of Windows emergency ;b

[grand anvil]

Then how can I start the app?

[supple venture]

Agreed it comes with everything one would need, and you can just install other stuff if you want, it is also a quite popular distribution so getting help is easy.

[supple venture]

sudo systemctl start test

[rapid sparrow]

the richest for good guides)

[supple venture]

👍 yeah

[grand anvil]

I have to replace test with my file name right?

[wooden ibex]

also, many companies require use of Windows/MacOS on Desktops

esp in the enterprise

[supple venture]

No with what you set the syslogodentifier as and the service name

[rapid sparrow]

in web dev I am usually seeing requirement to use Linux

like in every job position

[grand anvil]

Okay

[wooden ibex]

Web Dev are almost always MacOS at least in US

[rapid sparrow]

huh, specific of the country I guess

my country goes for Linux

[wooden ibex]

I guess if Mac are too expensive

[rapid sparrow]

definitely

[supple venture]

Haha indeed they are

[wooden ibex]

mean companies won't shell out for them

mine doesn't care dropping 1500 on MacBook Pro

[kindred copper]

Hello

Can I do some question about GitHub here?

[supple venture]

Sure go ahead

[kindred copper]

(Sorry for my bad english)

I clone a repository. It have 3 branches but in my working directory have only 1 branch (master)

Is normal?

And thank'u UwU

[maiden zealot]

Hello, does anyone have a recommendation for me? I have been using the Python pexpect module (https://pexpect.readthedocs.io/en/stable/index.html) to stream logs from a remote machine via SSH to regex search for events. I have wanted to reduce external dependencies (SSH), so I have converted the remote machine to stream these logs over a websocket.

Problem is, I didn't think about Pexpect not working "normally" with something that isn't simply dispatching a CLI command. Is there a similar module or something to pexpect where I could expect on an existing URI/websocket pipe?

[heavy knot]

this is docker rigth

[maiden zealot]

Looks like the docker desktop client to me

[heavy knot]

o ok

[lethal tinsel]

hey guys! My question in #help-potato went dormant and I thought maybe I could ask pip questions here. I've got a fresh venv pip install of docker 5.0.0 and it seems to say it requires websocket-client>=0.32.0 (which installs incompatible version 1.0.0) but their github and the package on pypi both say websocket-client==0.56.0 in their requirements.txt

Is there a place I can verify why pip thinks it needs >= 0.32.0?

[short peak]

https://github.com/docker/docker-py/blob/5.0.0/requirements.txt

not sure if that's the right place but you might want to take a look at that tag

oh you said that in your question, just ignore me please 🙏

did you see this https://github.com/docker/docker-py/blob/5.0.0/setup.py?

[floral smelt]

I have a question about a Docker Python image: at which moment are the environment variables defined?

[short peak]

for the containers you create?, on container creation, e.g. docker run --env VAR=VALUE python:3.9 python

[floral smelt]

Because I need to open a secrets variable at some points in my Dockerfile

Whenever I try to launch my app (FastAPI), os.environ.get returns none

Meaning I have to register the contents at some point in my Dockerfile somehow

[wooden ibex]

In Dockerfiles it's dockerfile env key=value

if you have multiple, do multiple lines

[floral smelt]

Yeah, my issue is:

• I have a secrets variable
• I want to extract the contents from secrets (a txt file)
• I want to put them inside a variable called DB_PASSWORD

[wooden ibex]

FROM tiangolo/uvicorn-gunicorn-fastapi:python3.8-slim
WORKDIR /app
RUN rm main.py
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY app/ app/
ENV base_url=http://micro_nginx-proxy_1
ENV MODULE_NAME=app.name_main```

[white tundra]

Hello guys, I'm trying to make a script to deploy automatically images, volumes, containers docker with python. Anyone did this until now? Also are you using subprocess for this task?

[wooden ibex]

There is ability in docker run to pass in environment variable file

[floral smelt]

It's not an .env file, but a db-password.txt that's already loaded into secrets

[wooden ibex]

which is just a txt file of KEY=VALUE on each line

[floral smelt]

secrets:
  db-password:
    file: ./db/password.txt

And what I want to do is extract that value for my backend (the database doesn't need it as it's directly stored in the appropriate variable)

[wooden ibex]

it's env_file for docker compose

[floral smelt]

Here's the issue:

  backend:
    restart: always
    container_name: fastapi-backend
    environment:
      DB_PASSWORD_FILE: /run/secrets/db-password

[floral smelt]

So should I do a duplicate file and store it in .env?

[wooden ibex]

you could write a script to pull info into env_file then pass that over

https://docs.docker.com/compose/compose-file/compose-file-v3/#env_file

[floral smelt]

ENV MY_NAME="John Doe" Can we launch bash commands after the =?

I want to do cat DB_PASSWORD_FILE

Except there's a scope with Docker variables

And if I do the same with a simple RUN command, I don't think it will end up as a container environment variable

[wooden ibex]

I think you have software design problem, if you want to read this information, you can always give this file to the container and let your python app read from it

otherwise, you can write a script to parse this information out of the file and write env file which you pass over before docker compose

[floral smelt]

Yeah because at some point I want to extract it for an ORM

from orator import DatabaseManager, Schema, Model
import os

pw_file = os.environ.get("DB_PASSWORD_FILE")


def read_secrets(fn: str):
    with open(fn, "r") as f:
        contents = f.read()
    return contents


DATABASES = {
    "postgres": {
        "driver": "postgres",
        "host": "db",
        "database": "example",
        "user": "user",
        "password": read_secrets(pw_file),
        "prefix": "",
        "port": 5432,
    }
}

db = DatabaseManager(DATABASES)
schema = Schema(db)
Model.set_connection_resolver(db)

[lethal tinsel]

yes thank you! I had someone else also point out that I should be looking at setup.py instead of requirements.txt. I opened a PR with them to adjust the version range for that dependency

[velvet spire]

@analog apex

[timber umbra]

try *site-packages/* without a slash in the beginning

besides that, 🤷

[velvet spire]

huh

now its

at 26

all my own

nope they're back

[velvet spire]

😳

why everything i do making it worse

[deep estuary]

where is your vennv

[deep estuary]

uhhh

why the heck is it

searching there

what folder do you have open in vscode?

[velvet spire]

    "folders": [
        {
            "name": "ModMail Bot",
            "path": ".."
        },
        {
            "name": "Plugins",
            "path": "../../modmail-plugins"
        },
        {
            "name": "Wiki",
            "path": "../../modmail-wiki"
        },

the full path to the workspace file is /home/pi/Projects/modmail/.vscode/modmail.code-workspace

and the workspace settings are just ```py
"settings": {
"python.formatting.provider": "black",
"python.pythonPath": "/home/pi/.cache/pypoetry/virtualenvs/modmail-oQfHkzPy-py3.7/bin/python",
"python.linting.banditEnabled": true,
"python.linting.banditPath": "/home/pi/.cache/pypoetry/virtualenvs/modmail-oQfHkzPy-py3.7/bin/bandit",
"python.linting.cwd": "/home/pi/Projects/modmail",
"files.insertFinalNewline": true,
"files.trimFinalNewlines": true,
"python.linting.flake8Enabled": true,
"python.linting.ignorePatterns": [
".vscode/.py",
"/site-packages//.py",
"site-packages/"
],

[deep estuary]

that's odd

you will likely find answers in https://discord.gg/b8YJQPx

[velvet spire]

🤔

now its not happening at the moment

wait

Ugh my internet sorry for the ghost pings

i uninstalled all flake8 extensions i had

and it still occurs

oh crap

will update u in a moment joe

…

@deep estuary its likely vsc insiders related

because i copied my config to stable

and the issue is gone

[deep estuary]

cnr on insiders, but maybe

[velvet spire]

i can't reproduce on not insiders

i installed poetry from the script and not through any pip

[velvet spire]

i mean yea lol. Now I can't get stable to launch zsh as my default shell LOL

👀

so far so good

spoke too sooon?

[velvet spire]

i rebooted the device. So far after 5 minutes of use it hasn't returned yet

[velvet spire]

…and it finally started to scan them again

[rapid sparrow]

There are several ways to handle secrets with docker.
Have your program extracting secret values from OS variables

import os
SECRET_KEY=os.environ.get("SECRET_KEY")

Preferably during config initialization, only one time

If you have only few values
You can put them into
docker run -e SECRET_KEY=65765, -e ANOTHER_SECRET=7477744 and et.c to launch your image

But better usually to store them in file like
.env.prod

SECRET_KEY=46546
ANOTHER_SECRET=56655
DB_PASSWORD=mysuperpassword

And get them all together with
docker run --env_file=.env.prod
(Rough example of syntax, it is slightly different, I wrote just how I remembered)

Anyway, os environment variables is the best way

Don't store in just simple text file

Have it in file like .env, which you never put into docker image.

Library python-dotenv does roughly the same if you don't have docker

Same can be done with docker-compose even in more comfortable way I guess.

But it will make just sugar to syntax which I showed you above already for docker run command

[supple venture]

You can fetch the repo to update your branches locally

[supple venture]

I would have used GitHub workflows for that, they seem the perfect option for this, where they would build and push the images on every commit you push. You could make the repo private if you don’t want it open sourced

[velvet spire]

how can they push if the computer doesn't have a public ip address?

or like one that stays static

[supple venture]

Make a repository and use GitHub actions on them.

[velvet spire]

finally, normal numbers again

[white tundra]

Yes, for sure can working with git. But I wanna do that only în py and from my hand to deploy some images containers volumes using certain commands. Also I will want to attach an interactive cli and autocomplete with arguments. That s my plan. I think first step is to read a bit about subprocess. :-?.

[supple venture]

Yeah subprocess would help out with this

[floral smelt]

Thank you! Yeah maybe I should switch to a common .env file instead. They seem to be more manageable.

[rapid sparrow]

u a welcome

[white tundra]

Anyone, do you think you would have time to help me to sketch something for the project with deploy img, container, volumes and cli with python? I don t have a clear ideea how I can do that and I will really appreciate if you can send me some help:) thanks!

[floral smelt]

Btw I have a question with Docker volumes

When I write that:

volumes:
  db-data: {}

Where are the volumes located?

Suppose we have this file:

version: "3.9"
services:
  web:
    image: nginx:alpine
    volumes:
      - type: volume
        source: mydata
        target: /data
        volume:
          nocopy: true
      - type: bind
        source: ./static
        target: /opt/app/static

  db:
    image: postgres:latest
    volumes:
      - "/var/run/postgres/postgres.sock:/var/run/postgres/postgres.sock"
      - "dbdata:/var/lib/postgresql/data"

volumes:
  mydata:
  dbdata:

Where is dbdata saved on the drive?

oh ok it's on a driver called local

But then it's saved on the Docker drive?

[supple venture]

which os?

[floral smelt]

Windows

With WSL 2

Also I was wondering if it was good practice to use a folder instead or not at all

[supple venture]

https://stackoverflow.com/a/43182885/14863850

[floral smelt]

ty

[candid kayak]

https://why.life-is-pa.in/0nx66R the wrangler CLI is just stuck at this step, anyone got any idea as to why?

[supple venture]

cloudfare wrangler CLI?

[candid kayak]

yes

[supple venture]

https://github.com/cloudflare/wrangler/issues/1703 this might help

[candid kayak]

i'm on that page right now

looks like setting it manually seems to work for others, i'll try that

[supple venture]

yeah, there are two three solutions listed on the issue, give them a try

[heavy knot]

how can i convert python file to exe, using pyinstaller and auto-py-to-exe error occured so please tell any solution

??

[wooden spoke]

use pyinstaller

[heavy knot]

getting error

[wooden spoke]

show me

[heavy knot]

failed to execute script

wait

[wooden spoke]

what happened

[heavy knot]

getting this error

can you help

??

[rapid sparrow]

Looking how cool Jenkins logo

I wish to try Jenkins

it is one of recommended must have anyway

[floral smelt]

When I was a beginner in Java, Jenkins was a pain to set up

[rapid sparrow]

well, but probably I'll get started with Ansible
Because I already use Gitlab/Github piplines and they serve as alternative to Jenikns
While I did not try any tool to replace Ansible... well, except self written shell + python scripts but I think it does not count

better would be to try things to which I have no alternatives yet%

Eh, so many things to try. Sometimes hard to decide

which wish to learn next

everything is needed

but what is more needed at the moment?

and how can I evaluate if it is more needed if some things can't be known you miss until learn%

[floral smelt]

I used Travis CI for testing

My friend also uses it

But idk if it’s sustainable. It works with Ruby, doesn’t it?

[heavy knot]

idk if this is off topic or not but when I use pyarmor to obfuscate my script into an exe how come when I open the exe it gives some import error??

it only happens when i import toast notifications

Traceback (most recent call last):
  File "<dist\obf\Faith.py>", line 3, in <module>
  File "<frozen Faith>", line 157, in <module>
  File "win10toast\__init__.py", line 127, in show_toast
  File "win10toast\__init__.py", line 93, in _show_toast
  File "pkg_resources\__init__.py", line 1144, in resource_filename
  File "pkg_resources\__init__.py", line 357, in get_provider
  File "pkg_resources\__init__.py", line 900, in require
  File "pkg_resources\__init__.py", line 786, in resolve
pkg_resources.DistributionNotFound: The 'win10toast' distribution was not found and is required by the application
[12512] Failed to execute script Faith```

thats the error with win10toast

[rapid sparrow]

oh cool. Nice to know at least one python code obfuscator, if I will ever need it. Thanks.

[bold roost]

Can I get help

[supple venture]

sure go ahead

[steady zenith]

Best GUI framework for someone coming from a webdev background?

or are they all equally very different from frontend webdev

[zenith sluice]

hello any ideas on how to communicate with an already running aws lambda function?
I was thinking to initiate a websocket connection from my lambda function is it possible? and keep it persisted

[supple venture]

kivy i had say, better suited for #user-interfaces

[steady zenith]

just did 🙂

thanks!

kinda scary timing

literally at the same time hahaa

[supple venture]

haha np!

[rancid schooner]

Hey @noble mountain!

It looks like you tried to attach file type(s) that we do not allow (.xlsx). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a.

Feel free to ask in #community-meta if you think this is a mistake.

[exotic fossil]

Anyone use Azure devops? Wondering if their pipeline supports building inside a docker container, defined using a Dockerfile

[brisk lake]

why does

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3
``` hang

on WSL2 ubuntu 18.04

im just trynna use poetry, why is this so goddamn hard

nvm it just decided to work

[wooden ibex]

Everything is built in docker container but yes Azure DevOps will build containers fine.

[random flame]

Hey, this is more of a github question, but is it possible to merge only certain commits from a pull request?

[exotic fossil]

@random flame You can cherry pick the commits instead of merging the entire PR.

@wooden ibex Do you have more info on this? Everything I've seen requires me to specify the container I want to build in using the image name, but I want to use a Dockerfile. The pipeline should build the image based on it, then build inside the container. Is that what you do?

@wooden ibex This link https://docs.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops has some examples but they all work by specifying an image name and not a Dockerfile

[wooden ibex]

Documentation is located here: https://docs.microsoft.com/en-us/azure/devops/pipelines/ecosystems/containers/build-image?view=azure-devops the image you specify is used as docker host and if for some reason you require certain versions of Linux for your docker host during build, you can specify that.

[exotic fossil]

@wooden ibex That is just for regular building of a docker image.
Do you know if a pipeline can be setup to build an image and spin up a container based on that image and run an app's build inside that container?

I don't want to publish the image to a container registry, want to skip that part.

[wooden ibex]

Err, why would you do that? BTW, you always self host a build agent if your build process is some crazy mess.

[exotic fossil]

The reason is I want to keep the Dockerfile stored in the application's repository and for every build I want to rebuild the image, that way if I'm making a lot of changes to my Dockerfile I don't have to run its separate pipeline, publish to container registry, update my app's pipeline to use the new container image, etc etc
And it also means I can easily maintain different build Dockerfiles on different branches. All self-contained in my app's source code repo.

I don't think the issue is whether the build agent is self-hosted, but really it's whether the azure pipelines YAML format supports this out of the box, or if I have to manually do it in my .sh script and just tell azure to run it

Jenkins can do this BTW, probably other CI/CD tools. It's not such a crazy idea IMHO 🙂

[wooden ibex]

Why do you want the build in a container? That makes no sense, build your project, get the artifacts and move on.

If your artifact is container, then push that to a repo

[exotic fossil]

To get the benefits of containerized build jobs, if I have C#, Java, Python, Perl, C++ etc. apps, then my build agent doesn't need to know that and doesn't need to have the build dependencies installed. The build agent only needs to have docker.

@wooden ibex ^

Of course it becomes easier to add new build agents; install docker, hook it up to the master node, and you're done, any containerized build can run on it.

If a build agent goes down, no big deal, the containerized build job doesn't care because it didn't need that specific agent, it can run on any agent with docker

[wooden ibex]

It’s the cloud, you always get a fresh build agent and thus your build steps must includes whatever dependencies besides the base OS and whatever language comes with build agent

[exotic fossil]

Ah but I'm not on the cloud 😉

[wooden ibex]

Unless you do self hosted build agent

If you are self hosted, that’s no handwave thing. That entirely different conversation

I’m sure you could write script and do some bind mount docker build.

Azure DevOps assume the build instructions source of truth is YAML file. If you want to do something different, I’d use different product.

[exotic fossil]

Yea, I was hoping Azure devops would do that for me. I do have that script, but if there's a simple yaml entry for it then I'd prefer that instead. Especially since its competitors can do it.

Ok, thanks for your help, it appears that there isn't a supported way by Azure Devops to do this (unless it's undocumented).

[wooden ibex]

Azure DevOps assume cloud agent. Their support for self hosted is a bridge to the cloud. If you want to operate more old school style, Jenkins sigh exists.

[exotic fossil]

@wooden ibex Hypothetically, even if I were on the cloud, what you said doesn't always work. Specifically, the fresh build agent that the cloud gives me does not necessarily include the build dependencies for my apps. They can't possibly have all the combinations of dependencies that everyone's apps needs. They can have a few basic/generic ones for a limited number of versions of frameworks/SDKs etc. but not everything.

So even on the cloud, containerized pipelines would be beneficial.

[wooden ibex]

Why? YAML file has all build instructions unless you are building a container which many do.

[exotic fossil]

Well, I would have to install the specific dependencies my app needs from that YAML file, which is more vendor lock-in. As opposed to defining those requirements inside a Dockerfile which can work almost anywhere else.

[supple venture]

Like raccoonbird said,

Say you committed to dev branch. And now you want to move it branch main. Say the commit ID is qwertyuiop. Then you can do something like this.

git checkout main
git cherry-pick qwertyuiop
git push origin main 

[mighty vessel]

Hey guys, I need help with GH and Git.
I forked a repo and then git cloned a different repo (the original repo of the one that I forked) but now that I'm trying to push to my fork, it's not letting me. Any ideas?

Ping me if you can help, thank you!

[supple venture]

Wdym by it’s not letting me, could show a screenshot/paste it here?

[regal ledge]

Hello! I have written a Flask application and it runs well on my local. However, I am struggling to find a place to start in order to deploy it live.

I have a FreeBSD server (with registered domain and such)that's all in my command but I don't know what to do. Must I do something in there to set a host, the domain and then do I need a venv, and are there any other steps that I need to take care of before I clone the code to the machine and try to run it?

Any links and pointers would be welcome. It's really my first touch point with app deployment/devops so basic truths are also welcome.

[neat meadow]

A couple of days ago, I migrated a project to VSCode and set it up with poetry.

Even though everything seems to be configured correctly (VSCode detects the poetry venv which points to the interpreter etc), whenever I try to run the Current File using VSCode's "run" button I get an sqlite3.OperationalError: unable to open database file error.

On the other hand if I just run it from terminal with python full_path_to_launcher.py it successfully executes the operation without the above error. Of course in this case I am not using the dependencies from the virtual env...

Any ideas on how to fix that or where might the problem lies ?

VSCode's settings json:

{
    "python.pythonPath": "C:\\path_to_project\\Cache\\virtualenvs\\testproject-Pz4YpBVZ-py3.8",
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true,
    "files.exclude": {
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/.pytest_cache": true,
        "**/__pycache__": true
    },
    "python.linting.flake8Enabled": true,
    "python.linting.enabled": false
}

VSCode launch config json:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal"
        }
    ]
}

[supple venture]

@mighty vessel sorry didn't see the "Ping me"

[velvet spire]

Are Code Owners on github repos only on paid repos?

[flat path]

@velvet spire IIRC "Code Owners" are available for public repos on any plan (even the free ones) but are only available on paid plans for private repos

[velvet spire]

ah cool

[velvet spire]

how to set up github to do this without having to manually add it?

https://i.imgur.com/Kye2kIA.png

[velvet spire]

since pydis gets pretty displays

[supple venture]

oh there would be a drop down just above that

shown

hiden

[grand gale]

there were commits on github that werent on your machine

thats because you removed those commits

[supple venture]

You need to do git pull like it says, since some of your commits are not tehre locally

one sec

u1 = "git add *"
u2 = "git push origin master --force"

After u1 you forgot to commit the repo

and before u1 you can always do git fetch origin and git pull so you are up to date with the git repo on github.

origin -> is your repo
fetch -> is to fetch a certain repo

Also i don't see the need to do --set-upstream, not really sure tho, never used it personally

fetch not fetc

also --force is not needed until you edit the already made git objects

and I am not sure about that, never used it personally

np 😊

[rapid sparrow]

wow.

https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow#:~:text=The overall flow of Gitflow,branch is created from master&text=When a feature is complete it is merged into the,branch is created from master

Awesome!

I think it is 42

[velvet spire]

It's additional checks

Its another work flow I think

[supple venture]

yeah they are different workflows

[supple venture]

https://github.com/Piston-CLI/piston-cli/runs/2653164604?check_suite_focus=true need some help here 😔

[manic mesa]

bruh

[frozen anchor]

Is there any way in GitHub to update the code diff on an open PR?

I created Branches A,B,C. B has all changes of A, C has all changes of B.

I then created PRs for all of them seperately.

PR1 has: Changes of A
PR2 has: Changes of A and B

Then I commited PR1.

What I would expect to happen:
PR2 (and 3) would update their code differences.
--> PR2 has: Changes of B (and not A anymore)

PR2 still has all Changes of A shown in the log - despite these changes already existing on MAIN - all of A still exists on MAIN, but it still shows the commits as differences.

Is there any like - "update" button to reflect the changes? I think this only makes it harder to review.

[supple venture]

Right, so say B has all the latest changes and you want to get those latest changes on A.

Then you can just rebase B into A, sbasically:

git checkout A # the branch which is behind
git rebase origin/B # rebase with the branch with the new changes

[wooden ibex]

Nope, merge C -> B -> A and A-> Main

and this is why branch strategy is important

[frozen anchor]

I am working with jira and the ticket id should be the same like the branch name. 3tickets -> 3 PRs -> 3 branches.
I do not understand how your plan would work on that.

[wooden ibex]

Because it doesn't matter what happened with A Branch is why Github is showing the diff. Even if main got modified, B Merge would change back any changes.

Thus why github is showing you the changes

fundamental issue here is your branching strategy

JIRA Tickets -> Branches is terrible idea, and why you are running into issues

[acoustic onyx]

Along with this I think it would great to discuss what is good branching strategy is!

I have setup few simple rules at my workplace -

1. main branch is protected and only accepts PR merges after at least 2 peer reviews.
2. Main branch is always buildable and never broken. Every merge is tagged as a release by following semantic versioning
3. Integration branch is taken off master and is routinely kept updated
4. All feature branches are made off the integration branch only. Branching from feature branch is not allowed. In case of blockers - get the feature branch merges first, figure out the management mistake that lead to such scenario
5. At required/scheduled time make sure integration branch passes all the tests, push database migration etc if required and then finally merge the PR in main

Branches are created on terminal and follow naming convention such as feature/<GitHub issue number>-issue-title

It's okay if it's refactor/123-foo or bugfix/567-foo

As long as it makes sense

[wooden ibex]

I recommend Main -> Dev -> Feature

[snow comet]

I think this is the right place to ask this question.... Im looking to buy myself a new laptop for on the go/ sofa coding. Im a long time Linux user but im curious about what apple has to offer recently, I see a lot of other programmers using macs. Is there anything in particular hardware wise that you lot can recommend?

Im not specifically asking for apple recommends by the way. Any laptops.

[acoustic onyx]

i don't think this is a right channel for this question

macbok air or xps 13 would do, right?

[plush rose]

Hey all, in atoms, I cant seems to find the window to show print() stuff when excuted,it's usually on the bottom of the screen.

Saw ppl doing it conveniently every time they do a print.

Kindly advise where to show that view screen.. thanks :D

[toxic bough]

you should ask your question in one of the off-topic channels 🙂

[snow comet]

Noted, thanks mate. Appologies.

[supple venture]

Just wanted to tell Lenovo laptops are great, you can look into them.

[tawdry needle]

Hard to go wrong with linux on thinkpad

Otherwise try to find an older macbook with no touchbar

They're so damn expensive

I wouldn't buy one again, I enjoy that i get one for work though

[snow comet]

I think ive come to the same conclusion over the course of the day. The think pad looks great.

[neat meadow]

If someone is accustomed with Poetry please elaborate on poetry config virtualenvs.create false --local option.
According to the docs
A) if virtualenvs.create returns false, the env must have pip available (duhh)
B) if virtualenvs.in-project defaults to None thus it will use .venv if it exists

Sooo... using the above mentioned option, inside the newly created projects, will poetry add work for dependencies or not?

[tawny temple]

I don't see why it wouldn't. You said yourself that the only requirement is that pip is available.

You can just try it and see if it works, if you're doubtful.

[digital moat]

@ionic pagoda what's your error

[ionic pagoda]

so i'm at the step of running python -m build, and this is the error (picture so you can see the file structure)

[iron basalt]

I have a T480 and an E485, they're great machines

[digital moat]

can we see your setup.cfg and setup.py

[iron basalt]

would recommend

[ionic pagoda]

setup.cfg

[build-system]
requires = [
    "setuptools",
    "wheel"]
build-backend = "setuptools.build_meta"

setup.py

from setuptools import setup

with open("README.md", "r") as file:
    long_des = file.read()

setup(
    name="Pypixels",
    packages=["pypixels", "pillow"],
    description="An API wrapper for the python discord pixels project",
    version="1.0.3",
    long_description=long_des,
    long_description_content_type="text/markdown",
    license="MIT",
    author="sebkuip",
    author_email="[e-mail]",
    url="https://github.com/sebkuip/pypixels",
    keywords=["API"],
    classifiers = [
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
        "Development Status :: 4 - Beta",
        "Intended Audience :: Developers",
    ]
)

i added pillow in packages as it's a requirement, this is the correct way right?

[digital moat]

I see the problem

you need to list the exported packages and dependencies separately

[ionic pagoda]

so packages= should only be my package name?

[digital moat]

yes

you can also use setuptools.find_packages()

assuming your folder structure is standard

pillow should be listed as install_requires

[ionic pagoda]

i copied from the tutorial

[digital moat]

in setup()

install_requires = [
        "pillow",
    ]

[ionic pagoda]

this is the new setup.py

from setuptools import setup

with open("README.md", "r") as file:
    long_des = file.read()

setup(
    name="pypixels",
    packages=["pypixels",],
    install_requires=["pillow",],
    description="An API wrapper for the python discord pixels project",
    version="1.0.3",
    long_description=long_des,
    long_description_content_type="text/markdown",
    license="MIT",
    author="sebkuip",
    author_email="[e-mail]",
    url="https://github.com/sebkuip/pypixels",
    keywords=["API"],
    classifiers = [
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
        "Development Status :: 4 - Beta",
        "Intended Audience :: Developers",
    ]
)

but still

C:\Users\*****\Desktop\projects> python -m build        
Collecting wheel
  Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Collecting setuptools>=40.8.0
  Using cached setuptools-57.0.0-py3-none-any.whl (821 kB)
Installing collected packages: wheel, setuptools
Successfully installed setuptools-57.0.0 wheel-0.36.2
WARNING: You are using pip version 20.3.3; however, version 21.1.2 is available.
You should consider upgrading via the 'C:\Users\*******\AppData\Local\Temp\build-env-6n4ppen3\Scripts\python.exe -m pip install --upgrade pip' command.
running egg_info
writing pypixels.egg-info\PKG-INFO
writing dependency_links to pypixels.egg-info\dependency_links.txt
writing requirements to pypixels.egg-info\requires.txt
writing top-level names to pypixels.egg-info\top_level.txt
error: package directory 'pypixels' does not exist

ERROR Backend subproccess exited when trying to invoke get_requires_for_build_sdist

@digital moat so didn't fix it

[digital moat]

that looks better

did you name your folder src\pypixels?

[digital moat]

it should just be called pypixels

in the same folder as setup.py

you should have a folder called pypixels

and then a python package in that

[ionic pagoda]

I copied this over pretty much

[ionic pagoda]

uhhh..

so no src folder? as instructed on pypi's tutorial?

[digital moat]

well the way I do it is to just have the package folder in the top level directory

if you use setuptools.find_packages()

instead of listing them manually

then your way would work

you could also specify your path manually

right now you have packages=['pypixels']

if you did somethign like packages=['src/pypixels']

maybe that would work

[ionic pagoda]

wait could it be that in my packages= it just says pypixels and not src/pypixels (or whatever it is)?

[digital moat]

but I'm not sure

[digital moat]

yeah

exactly

[ionic pagoda]

that did the trick lmao

[digital moat]

using packages=setuptools.find_packages() would be the easiest way though

[digital moat]

nice

[ionic pagoda]

for now i'm just following pypi's tutorial

[tawdry needle]

@ionic pagoda

from setuptools import find_packages, setup

setup(
    ...
    packages=find_packages('src'),
    package_dir={'': 'src'},
    ...
)

[dire saddle]

hello there

can anyone assist with pycharm? im trying to set up my mac and its showing "no python interpreter" even though i have idle installed

[supple venture]

Need some docker help
code: https://github.com/discord-modmail/modmail/commit/97f7ddd2832b9da09a58cfd209b3c48ab9ca1c84

The problem I am facing

[velvet spire]

🤔

[velvet spire]

what's right above that?

[supple venture]

ValueError

[velvet spire]

lemme try

:crazy:

Failed to build cffi

[ionic pagoda]

@digital moat not sure if you're available right now, but when i pip install i need to do from src import pythonpixels (had to rename due to name taken). Now i changed the directory to no longer have the src/ file, and pushed an update. But still requires src/pythonpixels?

do you by any chance know how one could solve this?

[digital moat]

what do you mean by still requires

[ionic pagoda]

it's currently this

and setup.py packages=["pythonpixels",],

so removed the src there too

[digital moat]

does it work ok then?

what's the error

[ionic pagoda]

but when i do import pythonpixels it's unknown

when i do from src import pythonpixels it works

whilst src doesn't exist anymore

[digital moat]

yo're in a different directory and you've installed the latest version from pypi?

[ionic pagoda]

yes

oh i might see, it might be the build files from previous version

[digital moat]

ok

[ripe junco]

I have a Flask app running on a Docker container, but when I click the address generated by Docker, it does not found my home.html template, even tho it is inside of the templates directory. How can I fix it?

Here's my Dockerfile

FROM python:3.6.9

ARG apache_version=apache2

RUN apt-get update && apt-get install -y \
    software-properties-common \
    ${apache_version} \
    curl \
    git \
    python3-pip \
    nano


RUN pip install --upgrade pip
WORKDIR /home/modsi
COPY ./requirements.txt /home/modsi/requirements.txt
RUN pip install -r requirements.txt

#Copy necessary files to environment
COPY ./db /home/modsi/
COPY ./database_conn.py /home/modsi/
COPY ./app.py /home/modsi/

ENTRYPOINT python3 app.py

[urban pecan]

i'm building an import-alternative and would like to implement an auto-install feature that would work something like np = use("numpy", version="1.19.2", auto_install=True, hash_value="98d78868665") that would take advantage of the hash-checking mode of pip, is there any best-practices that should be included as default (i'm thinking of using a virtualenv etc)?

[ripe junco]

And my docker-compose

version: "3.0"
services:
  database:
    image: postgres:11
    container_name: modsi_postgres
    env_file: 
      - .env
    environment:
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DB}
    ports:
      - 5432:5432
    networks:
      - db_modsi
    volumes:
      - ./modsi_data:/var/lib/postgresql/data/
      - ../db/create_db.sql:/home/modsi/create_db.sql
      - ../db/automate_db_creation.sh:/home/modsi/automate_db_creation.sh

  modsi_app:
    build: ../
    ports:
      - 65200:65200
    networks: 
      - db_modsi
    depends_on: 
      - database

networks:
  db_modsi:
    driver: bridge

Here's how I run the Flask app

if __name__ == "__main__":
    Timer(1, open_browser).start()
    app.run(debug=True, host='0.0.0.0', port=65200)

[tawdry needle]

@ripe junco did you forget to COPY the templates directory?

[ripe junco]

Yeah, I've already solved it, the db and templates directories weren't being copied completely

Thanks for the help tho

[tawdry needle]

venv is tricky because you don't have control over which python the user is using to run their code

also doesn't pip not have a public python api?

[plush thorn]

So I am on windows using vsc code and I created a virtual environment named enivronment_3_9_2 but when I run this command to activate it...it doesn't work.....environment_3_9_2/bin/activate it throws following error:

cd : Cannot find path 'E:\Django Projects\DJANGO_ECOMMERCE\enviornment_3_9_2' because it does not exist.
At line:1 char:1

•   + CategoryInfo          : ObjectNotFound: (E:\Django Proje...viornment_3_9_2:String) [Set-Location], ItemNotFoundException
    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.SetLocationCommand
  
  

PS E:\Django Projects\DJANGO_ECOMMERCE> environment_3_9_2/bin/activate
environment_3_9_2/bin/activate : The term 'environment_3_9_2/bin/activate' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1

• environment_3_9_2/bin/activate

•   + CategoryInfo          : ObjectNotFound: (environment_3_9_2/bin/activate:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

[dense grotto]

I have been working on a proof of concept script that uses paginated HTTP REST API calls and it's working out very well. I've ended up using a number of Python3 modules in a virtualenv on my development system. I'm now at the point where I want to be able to make this available to colleagues. I haven't had to worry about sharing requirements, packaging, and virtualenvs before. I'd like to figure out a good way to be able bootstrap my colleagues (who are not Python users/developers) so that they can use what I've written. I'm finding it hard to figure out what tools/techniques to use. Any pointers?

[urban pecan]

@tawdry needle that's why i ask here 🙂 since i'm not deep into devops so far, i'd appreciate some pointers for current best practices in that regard

how would you realize a script that utilises pip in hash-check mode with a virtual env? is there a better way that gives the same result?

i don't really care about the details at this point, just the result - it should be as automatic as possible with the name of the package, version and hash given

did you mention the pip python api as recommendation?

@dense grotto would you like to try something i've been working on? it could be a possible solution 🙂

https://github.com/amogorkon/pyuse/blob/main/src/use/use.py - they could use() your stuff - it should work with installed packages but also with local paths and URLs (like on github)

it's a single module with few dependencies (mmh3, anyio, requests, yarl)

i didn't get to packaging it to pypi yet (it would be easier if there wasn't a "use" package already..)

most of the stuff works for my own scenarios, so feedback would be helpful at this point

you could get the raw link for your stuff on github, put it in use() on your side, note the hash, share both, link and hash with your coworkers, then they could simply do module = use(use.URL(rawgithublink, hash_value=your_hash))

default hash is sha256, which is standard for pip as well

[rotund remnant]

Anyone have thoughts on ways to specify the python version when using pip-tools? I've been using a pip-compile based workflow for a while and generally like it. The one big wart is that the generated requirements.txt file is sensitive to the python version used to create it and there is no record of which version was used by default. I can think of a few ways out, but looking for opinions on how other people handle this.

[shy mountain]

Can someone please assist me on creating a Dockerfile to use some computational chemistry libraries? I've been in dependency hell for weeks. Seemingly circular dependencies, etc.

Here is my script: Currently just looking for files and reading them as molecules.

# this is a script to grab XYZ data
import os
from chemml.chem import Molecule
import importlib
import sys
import rdkit

def traverseFiles():
    rootdir = '.'
    for subdir, dirs, files in os.walk(rootdir):
        for file in files:
            filePath = os.path.join(subdir, file)
            if(("gs" in filePath) and 
                ("Meta" in filePath) and 
                ("xyz" in filePath)):
                mol = Molecule(filePath, 'xyz')
                print(mol)
traverseFiles()

Here is the Dockerfile I am using currently.

FROM continuumio/miniconda3:latest

# Update system
RUN apt update -y
RUN apt install -y gcc

# Install Conda dependencies
# RUN source activate pipeline
RUN conda install python=3.6 # I thought python 3.2 was required for openbabel?
RUN conda install -c conda-forge rdkit # need conda because there's shared libs to install
RUN conda install -c openbabel openbabel # have to use this in order to install openbabel 2.X
RUN conda install -c anaconda tensorflow

# Download and instal most recent version of chemml
# RUN git clone https://github.com/hachmannlab/chemml.git
# RUN pip install -e ./chemml
RUN pip install chemml

# Create working directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

# Copy contents
COPY . /usr/src/app

# Run script
RUN python3 driver.py &

If I pop a shell in the container and try to run the script, I get the following error.

Traceback (most recent call last):
  File "driver.py", line 3, in <module>
    from chemml.chem import Molecule
  File "/opt/conda/lib/python3.6/site-packages/chemml/chem/__init__.py", line 14, in <module>
    from .molecule import Molecule
  File "/opt/conda/lib/python3.6/site-packages/chemml/chem/molecule.py", line 5, in <module>
    from openbabel import pybel
ImportError: cannot import name 'pybel'

Ok, so I try pip install pybel. Same error when running the script. I can run a python shell and import pybel, but chemml is weird? I just dont know how to fix it

[dense grotto]

@urban pecan I will look. I should probably have said that the intended way to run the script is from the command line (in macOS). I'm primarily worried about getting Python3 installed, getting a virtualenv set up, and getting the required modules installed. I'm currently reading more about poetry.

[urban pecan]

okay, then it probably won't fit your bill

[heavy knot]

i have to be voice verified

Kinda weird but okay

[short peak]

@dense grotto . I use pyenv to switch to python3 and use poetry on macOS without 2.7 warnings

[bold frigate]

Can anyone please help me with heroku deployment? I want to deploy my python app, also have created heroku app, added Procfile, requirements.txt, setup.sh files to my working application repository and have also installed heroku client. Need further assistance!

[heavy knot]

I have this code

When I run it, with python time_split.py song_list big_file.mp3 "Martin O'Donnell,Michael Salvatori" "Halo: Reach OST"

It gives me an error:

Traceback (most recent call last):
  File "/home/ryuo/Downloads/Youtube to mp3 splitter/All minecraft soundtracks/youtube-mp3-split/sample_run/time_split.py", line 94, in <module>
    main()
  File "/home/ryuo/Downloads/Youtube to mp3 splitter/All minecraft soundtracks/youtube-mp3-split/sample_run/time_split.py", line 44, in main
    song_type     = sys.argv[5]
IndexError: list index out of range

[rancid schooner]

Hey @heavy knot!

It looks like you tried to attach a Python file - please use a code-pasting service such as https://paste.pythondiscord.com

[heavy knot]

https://paste.pythondiscord.com/guxopezido.py

[runic verge]

when youre looking for assistance i see the best way is to do #❓｜how-to-get-help and start a help channel ... post what youre looking to do .. and then tag your help channel here for people to move to

i just checked this out, i haven't attended before either , i see it'll be interesting

[runic verge]

i've handled this by using different virtual environments that have a different python version installed . I use anaconda for this

[runic verge]

I wonder whether anyone has had security concerns while using docker .

i don't know docker well , so unfortunately im not able to help with this

[runic verge]

if __name__ == "__main__":
    Timer(1, open_browser).start()
    app.run(debug=True, host='0.0.0.0', port=65200)```
i am wondering which file this code is in , and how it is communicating with docker . i also wonder whether port 65200 is specified as docker . And what is open_browser ?

[runic verge]

i see this is a project with multiple questions to it .. which i see is why nobody gave a response . it's a complex solution that can't be explained in a few minutes . This is usaully what i see companies pay for . A simple question is how to set the application live , thats for google . When i read that you have a server .. then use the options for freeBSD . Hearing that you dont know what to do , doesnt give us confidence in knowing what you want to do . Can look up each of the other questions in google and then come back to us to clarify a solution .

I've heard flask is preferred for development .. and heard its better to use something intended for production when doing a wide-scale deployment . I suggest reviewing more about this as i've read it can cause a bottleneck in the APIs' ability to process multiple requests together

[heavy knot]

Anyone help ..
y is 11 then why it is telling that x is greater

???

[agile obsidian]

this probably isn't the right channel for that, check out #❓｜how-to-get-help :)

[heavy knot]

bruh it will take time

[agile obsidian]

mmhm, we're all just volunteers

but you're certainly not going to get a faster response when asking a question like this in the #tools-and-devops channel

[heavy knot]

okay okay

[runic verge]

I am wondering how to contribute to open-source code on github. i have given credit to another project on github .. in my readme , and i've created a fork of it . but i wonder how code is actually contributed to a main repository. Basically i see this would be going back and forth with someone that is still actively working on a project

[orchid ore]

The process is called a Pull Request or PR, basically you edit your fork, and then create a PR in the main repositorry which is asking them to use your changes

[tawdry needle]

i think you'd have to use the python version specifier in requirements.in, e.g. importlib_resources>=1.13; python_version<'3.9'

[tawdry needle]

no, i'm pretty sure there is no such api, at least not with any public docs or stability guarantee

[tawdry needle]

you probably know more about pip than i do in this area... i'd recommend falling back to pip install --local if the user isn't obviously in a detectable virtualenv, venv, or conda env

it gets messy because maybe the user is in an isolated pyenv installation and doesn't want to use --local

maybe you can simply fail if the env isn't detectable, then add "i am sure i know what i am doing" options

# Use current python interpreter if venv/virtualenv/conda/mamba env is detected
use("https://git.sr.ht/~foo/bar")

# Use current interpreter, without env detection, but using `pip install --local`
use("https://git.sr.ht/~foo/bar", install_local=True)

# Use current interpreter, with plain `pip install`
use("https://git.sr.ht/~foo/bar", install_system_unsafe=True)

im not sure about the hash checking component... how are you getting the hashes?

[urban pecan]

https://pip.pypa.io/en/stable/cli/pip_install/#hash-checking-mode

well, regarding github, this already works: utils = use(use.URL("https://raw.githubusercontent.com/PIA-Group/BioSPPy/7696d682dc3aafc898cd9161f946ea87db4fed7f/biosppy/utils.py"),
hash_value="77fe711647cd80e6a01668dba7f2b9feb8f435ba")

[tawdry needle]

yeah, i know that much. but i also know that e.g. pip-tools can fetch hashes to build its requirements.txt file, presumably from pypi

[urban pecan]

if you don't give it a hash_value, it spits out the one it got, then you can reuse that later

[tawdry needle]

thats useful

do you just use subprocess to invoke pip?

[urban pecan]

that was my idea, yeah

i never tried to work with pip hash mode, so i was hoping to get some thoughts on that here

[tawdry needle]

being "in" a virtual env is kind of irrelevant from that perspective i think

unless you want to have different hashes depending on platform, py version, etc.

[urban pecan]

my idea was to first check if the package already is installed, if not build a venv or something similar and pip install the package there

[tawdry needle]

well you can't build a venv around an existing python interpreter

[urban pecan]

hm..

[tawdry needle]

i'd do it this way:

if python_is_in_venv():
    ...  # python -m pip install foo --hash=abcd
elif install_local:
    ... # python -m pip install --local foo --hash=abcd
elif install_system_unsafe:
    warn("This is dangerous, you're on your own.")
    ... # python -m pip install foo --hash=abcd

[urban pecan]

is there a way to build a temporal, in-memory venv maybe just for this session?

[rotund remnant]

Yeah, for cases where the dev knows that info. But I want that captured automatically in the compile step. Poetry, e.g., seems to do it nicely. I am thinking a relatively simple pip-tools enhancement may be to use the same environment tag system that wheels use

[urban pecan]

temporary*

[tawdry needle]

no but you can download the package to some temporary location and then append that location to sys.path @urban pecan

[urban pecan]

that might work, yeah

is there something i need to take care of? cleanup?

[tawdry needle]

that im not sure of

maybe there's a way to manually tell pip where the target site-packages should be

im not sure how pip and setuptools interact internally

[urban pecan]

temporary packages.. it's getting crazier and crazier ^^

[wooden meadow]

boto3 question anyone?

[urban pecan]

@tawdry needle any suggestion on how to check if python is running in a venv?

[tawdry needle]

@urban pecan i believe there's an env var PYTHON_VIRTUAL_ENV or something like that

i believe conda has its equivalent CONDA_DEFAULT_ENV

[urban pecan]

thanks

[tawdry needle]

need to check to confirm

stackoverflow knows

[urban pecan]

nod

auto-installing packages, auto-reloading, web-sources.. if there was something like a python inquisition, i'm sure i'd get burnt by the stake ;p

[pseudo moon]

you might get some inspiration from tox and nox as far as manipulating packages & environments

[rotund remnant]

Maybe there is inspiration in how pep517 build isolation works

[tawdry needle]

or poetry, which creates venvs temporarily

[urban pecan]

thanks, i'll look into those 🙂

[short peak]

I like poetry a lot 🙂

[urban pecan]

would you recommend poetry as best practice for automating this kind of stuff?

[short peak]

sorry, kinda lost. What stuff?

[urban pecan]

heh.. nvm ^^

[tawdry needle]

no, poetry is a different tool for different tasks

i recommend sticking with pip and setuptools since they're (usually) shipped with python now

certainly the official python.org packages include them

[rotund remnant]

I think it's just VIRTUAL_ENV

[tawdry needle]

you're probably right, good catch

[heavy knot]

What is the difference between Django socket.io and Django channels?
what to use when?

[short peak]

there's a #web-development channel. Perhaps you can try there

[sand epoch]

Any server related to LINUX??

[bold frigate]

Noted @runic verge

[runic verge]

was wondering how many companies utilize containerization for their projects .. i had just finished watching the Dockercon convention and saw that the CTO went through about how their operations can be hacked . I have talked a lot about security and decompiliation/ tracking of what an application is doing .. in the discord . Has been very interesting to hear their take on their own development environments security vulnerabilities . I haven't used docker , after a review of how the system operates and saw the similarities to what they mentioned and what i found. In that i see i have done a lot with devOps . but not sure where that can be integrated with opensource projects .. when that's actually the analysis of open-source projects to verify they are secure before integrating them into a project

[runic verge]

also wonder whether i am doing this right for github. https://github.com/jithurjacob/Windows-10-Toast-Notifications/issues/18 I posted here , been having this issue with this library for quite a while , and i havent seen it fixed. for a few years it seems like it has been a problem . Its kind of documented on stackoverflow well , no definite fix though , but theres not a lot of notice to the topic . So i do see the opportunity to fork it and ask for a PR when the issue is resolved. I am to the point where i am modifying the packages files in order to change it to get it working ... i see that is a reason to PR it . @civic hound @orchid ore Also wonder whether i could invite these two to work on the fork as a group , and we commit the PR together.

[wispy cliff]

I am new here anyone there?

[runic verge]

yes we are here

ask away

[wispy cliff]

ahh i am new to python just downloaded pycharm yesterday what to do?

[runic verge]

generally i see this conversation is better for the #python-discussion channel. becuase there are a lot more users that "hang out " there frequently . and this channel is meant specifically for tools-and-devops

i see they may suggest different projects for you to try

[wispy cliff]

ok

[runic verge]

tag me there too

[rapid sparrow]

Deployment time of my project from whole day got decreased to some minutes with docker

And lessened documentation from dozens lines to few commands

I can't imagine my life without docker

[runic verge]

I’ve heard this .. and seen it . Just when I am seeing this picture of the storage containers getting loaded onto trucks , I see personal data and company files shipping to other companies and docker itself . The squid adds to it.

[rapid sparrow]

docker can be used privately

I build images on only where I am alone having access

[runic verge]

I consider running docker on a separate company computer

[rapid sparrow]

I don't send them to remote storages

[runic verge]

Yes well, that is safe from cloud processing , but I’ve found more insecurities than what’s relevant to cloud

Docker operates at the same level as an operating system and what it’s actually doing isn’t monitored, but also connects to the internet to download packages and things . What we see on our operating system is basically like a report from docker of what it says it’s doing. We don’t see what it’s actually doing . I haven’t found a way to completely monitor docker to know for sure that it’s secure .

[short peak]

is dockercon 2021 finished? Any highlights?

[runic verge]

Haven’t seen highlights . Dockercon finished at 8-9 pm est yesterday .

Was an event that was registered for . Wonder whether you can register now.. I’ve seen the videos are still up for playback

@short peak

[short peak]

any interesting topics you can highlight?

[runic verge]

Also docker has access to all of the bare metal components that the rest of your computer operates on .. hard drives being the main concern .

[runic verge]

The CTO came on and talked about the insecurities relevant to docker when connecting to the cloud . Some of them I saw were very similar to the insecurities off cloud. And generally hearing that docker is not ready , or secure . I was primarily interested in what was said about security though

I see docker is amazing , I also see that it’s a large security risk for anyone that is not running it on a dedicated machine . And more so for cloud

[finite fulcrum]

That doesn't seem as relevant when you're only running trusted programs on it, like when deploying your own code

[runic verge]

The benefits for businesses I see far outweigh the cons.. which are that data is easily stolen , but I see that wouldn’t go noticed easily . And also i see means lots of business are going to hop on board. Was wondering whether anyone’s company has utilized docker, that is in the docker . And might like to talk about it

[runic verge]

One of the things I heard them talk about was whether “trusted docker images” were completely secure . The response I heard is that they are more secure because it comes from a trusted source , but once an infiltrator is in, they have the ability to install any other tools they need in order to do whatever hack stuff they are planning to do. Heard that it’s just an extra step.

Specifically about using docker on the cloud

I wonder whether there are going to be highlight videos on YouTube from dockercon. It’s interesting to hear from the owners, CEOs , CTOs of the company . Dockercon was streamed on their site so idk whether they’ll be released outside of the event

[finite fulcrum]

I personally don't see that as much of a docker (program) issue, as it applies to any program that may run on the machine, docker would just be an intermediary there

When it's being ran in the development environment I'd say there are more potential security issues there and most of what will run in docker will also be ran there

[runic verge]

From how I’ve seen docker , and heard docker talked about , it’s not a program . It’s more of a downloadable development environment. And it has large security holes . So it has the potential to do more than a program .. program having a defined limit of ability. Docker also operating on bare metal

Not an application on an operating system either , it is the operating system . And on our operating system (the one we see , maybe windows 10) we see a report of what docker says it’s doing inside the operating system that runs parallel to windows 10. And we don’t see what’s happening on the docker operating system , we get a report from docker . So is a question of trusting the most insecure, and capable thing on the internet, for me.

Can analyze what docker sends and receives over the network by monitoring a router.. the important data is encrypted with HTTPS .. when you have the key that it was encrypted with , the data can be decrypted . But still some data may further be encrypted within a program . Which means we wouldn’t know what data is sent and received even with the “encryption code “. Because these may be another encryption code between the servers backend and the compiled program that would be decrypted upon reaching the (dockers’) server. To see that the program I see would need to be decompiled and that’s not fun or easy at all. Same for most other applications. Even small applications that are used for 5 minutes, but would take 5 months to decompile.

[tawdry needle]

I think "trust" comes down ultimately to who you think you can trust.

Do you trust the Debian maintainers? Do you trust that their FTP servers are secure? Do you trust Docker the company? etc. etc.

At some point you have to trust someone, and use defense-in-depth when possible. (Caveat: I am not a security professional)

But if your organization is really concerned about security (e.g. you store customer PII) you probably should hire a security professional. Instead of guessing!

[runic verge]

Docker is different from a virtual environment in that a virtual environment runs on your operating system . Where docker actually is the operating system... and a main benefit of docker is that it has very low overhead for processing power .. first thing I notice is that it doesn’t have a GUI and all the extra stuff that windows has .. which relatively takes up a lot of storage and cpu processing . I look at docker as a computer that is running on the same hardware that we connect to via a very local network. And when we give it a docker image it is a complete operating system . Where before docker was something that allowed an operating system to be docked.

[runic verge]

The interesting thing I see about defense in depth when it comes to docker is that it is designed to be separated in ways that anyone can download the dockerfile and run the files . So the indexing is very good .. and I see depth wouldn’t be as time restrictive

[tawdry needle]

In this case, the "depth" provided by containerization is isolation from other parts of your software infrastructure.

However I've been told repeatedly that the container technology underlying Docker does not constitute a sandbox for security purposes.

Docker is a deployment tool, providing dependency isolation, flexible deployment, etc. but not additional security.

[runic verge]

Docker is an operating system that communicates on a network

The security is supposed to be within it

[tawdry needle]

Right. So it's no different from any other Linux (or whatever) deployment.

[runic verge]

I see docker has more capabilities as an operating system.. and more access to the other operating systems operating on the same bare metal components than either windows or linux does . But I’m not sure as to the full capabilities of linux and windows.. but I see especially windows is going to be more rigid in what it can do. Thereby providing security

Most users I see don’t understand the capabilities of docker because it is so easy to install . Wouldn’t know that it actually runs as an additional operating system unless you’ve looked . Seeing it’s underplayed. And I see other applications aren’t exception to being underplayed to what they have access to .. just other applications don’t have the access that an operating system does . (Google chrome I’ve looked into before and it’s similar in being underplayed . Ad-Blocker too)

Ad-Blocker having a full search history.. and some connections to the file system

[rapid sparrow]

I just run docker exec -it containerID apt install mc
and then enable my GUI with dcoker exec -i containerID mc

[tawdry needle]

and more access to the other operating systems operating on the same bare metal components than either windows or linux does
i'm not sure what you mean by this

[rapid sparrow]

I wonder if there are some more console friendly alternatives to mc

oh... right, I'll just need to install openssh-server

and connect to container

it will be I guess more handy than mc

[tawdry needle]

there are other TUI file browsers like nnn

also a lot of graphical file browsers can connect with sshfs or sftp

or yeah, if it's docker you can docker exec /bin/sh or something like that

[wooden ibex]

Containers is heavily used in many companies

[ebon swift]

I'm installing a CLI built in Click from a private repo using 'pip install git+ssh://[e-mail]/xxx/xxx.git#egg=xxx'. How does 'pip install --upgrade xxx' work in this context? Essentially I want to create a package in a private repo that can be upgraded via pip, but I can't find anything on Google about this.

[tawdry needle]

it would work fine. pip clones the repo no matter what, because it doesn't know how to get package version information otherwise. then the behavior of --update is the same as usual. if your machine can't access the git repo (e.g. because you don't have the right ssh key) pip would fail to clone the repo and it would report an error

[grizzled raptor]

Hi, I'm trying to build a wheel of my package to run on azure. However, I think the azure compute clusters run on AMD and my local system is x86. Does anyone know how I can cross compile a wheel on AMD?

It has a native extension so I can't just build an universal wheel

I think

[wooden ibex]

Umm, do you not have 64 bit processor?

[grizzled raptor]

I do, I meant my processor is intel x86_64

[wooden ibex]

Ok, AMD and Intel are interchangeable with rare exceptions

Like what are you doing that makes your 64 bit wheel not work on AMD

[grizzled raptor]

@wooden ibex I have a native extension that I assumed wouldn't work on amd

Especially since the wheel's name ends in "x86-68-linux-gnu"

[wooden ibex]

AMD owns patent on x86-64

[grizzled raptor]

oh I think I heard of something like this

So my wheels should work on both AMD and intel in that case?

[wooden ibex]

Yes

[grizzled raptor]

That's good to know, thanks

[grizzled raptor]

I've run into another issue: My compiled wheel apparently requires a glibc version that is newer than the one used in the docker containers I need

Is there a way to compile wheels for older versions of glibc or is there maybe a way to acquire a newer glibc version through e.g. conda?

[tawdry needle]

use the same docker base image to build the wheel

but yes i think conda default or conda forge should have some different gcc and glibc versions available

[urban pecan]

btw, i managed to wrap up use() as a package. now it should be possible to python -m pip install justuse and then import use to import stuff from anywhere. it dispatches to str for regular packages and builtins, pathlib.Path (use.Path as alias) for local modules and yarl.URL (use.URL as alias) for web sources

it's also possible to pass in module-globals on import

which probably can simplify more complicated systems

i'd love to get some feedback, ideas on how to improve it etc.

possible usecases that aren't covered yet

[supple venture]

Got a question, about packaing, particularly for this package: https://github.com/Piston-CLI/piston-cli/

Will the files included in the repo root be included in the package, like If i add Nix flakes will the package take them?

[prime grail]

Any way of having cumulative graphs in Grafana? As in the graph acts like a counter in which the values will only go up.

[tawdry needle]

post the link again? could be an interesting project to share on r/python, discuss.python.org, etc

[tawdry needle]

not by default. this project uses poetry, which by default includes all files inside a python package (this behavior is somewhat different from setuptools). but if it's not already inside a python package, you will probably have to add it explicitly to the pyproject.toml file https://python-poetry.org/docs/pyproject/#include-and-exclude

e.g. if you wanted to include a NEWS file in the build, you'd write include = ['NEWS']

[heavy knot]

i have problem pip install away dont work

I got errors

[grizzled raptor]

That was a good point and is what I ended up doing - thanks

[urban pecan]

@tawdry needle https://pypi.org/project/justuse/ https://github.com/amogorkon/justuse

there's still much to do until i've got everything in what i've planned, but the basic functionality works for my usecases

i'm pretty sure people in an enterprise setting will hate me for that 😉

but it might be useful for researchers etc. working a lot with jupyter

or devs who need to quickly prototype something..

earlier today it gave me a heads up on an version upgrade of hdf5 that happened while installing keras

which is quite useful because it might mean that i need check previous API breakages

[tawdry needle]

@urban pecan why put a class in as a module?

Can't you do import use._use; sys.modules['use'] = use._use or something?

[urban pecan]

i wanted to have the interface as simple as possible, making the module callable similar to how q does it seemed the most direct approach

replacing the module with a callable class

q does more weird stuff, i tried to keep the weirdness at a minimum ^^

my first approach was to replace the module with a single function, but with more cases to handle, it grew too complicated

also, i'm thinking of applying some syntactic sugar like @ for module-decorators or similar