#tools-and-devops

1 messages ยท Page 52 of 1

sand thistle
#

lol

gusty willow
#

lol

sand thistle
#

i went through the logs but it's all gibberish to me

#

the ufw logs

#

I see a bunch of UFW blocks

gusty willow
#

what happens if you just disable ufw temporarily

#

obviously, don't leave it that way, but that would eliminate your firewall as the actual problem or not

sand thistle
#

well i disabled it

#

and I tried to reach it

#

but it still didn't work

gusty willow
#

ok, so that eliminates UFW as the problem

sand thistle
#

hmm

#

so thats weird

#

i restarted the py file

#

and now i can access it on the web

gusty willow
#

and does it quit if you turn UFW back on?

sand thistle
#

well I did already

#

so no

#

I think it's the wsgi entry point

gusty willow
#

ok, well good ... we have eliminated a potential cause of the failure ๐Ÿ˜„

sand thistle
#

or gunicorn

#

because if I stop runnign the script

#

I can't access it on the web

gusty willow
#

how many workers are you running?

#

because i've seen it to where not all of the workers were able to access the entrypoint before (and that is some weird stuff to troubleshoot, let me tell you)

sand thistle
#

but arent they separate entry points

#

hmm so rebinding gunicorn helped

#

well

#

fuck

gusty willow
#

lol what now

sand thistle
#

nothing works

#

I want to cry

gusty willow
#

just fyi, be mindful of the language

sand thistle
#

k

#

yeah idk

#

my systemd file is messed up or something

#

instead of using a unix socket though

#

I'm using tcp

#

and doing localhost:8080

gusty willow
#

i mean, that should work

sand thistle
#

yeah

#

but it doesnt

gusty willow
#

you can browse it locally?

#

curl http://localhost:8080/

sand thistle
#

connection refused

#

systemctl status shows that there is a pathing issue

#

Failed to execute command: No such file or directory

gusty willow
#

well ... if you can't even do the localhost curl, then gunicorn isn't running

#

are you trying to run gunicorn with systemd?

#

i may have missed that bit if that is the case

sand thistle
#

i believe so

#

Description=Gunicorn instance to serve myproject

gusty willow
#

lol sounds like it

#

what does that whole thing look like

sand thistle
#

the systemd file?

gusty willow
#

yeah

#

my guess is you probably just need to provide the full path to the gunicorn executable

sand thistle
#


[Unit]
Description=Gunicorn instance to serve expDashboard.py
After=network.target

[Service]
User=expadmin
Group=www-data
WorkingDirectory=/home/expadmin/ExpFlask/managment_dashboard
Environment="PATH=/home/expadmin/ExpFlask/managment_dashboard/expDashboardVenv/bin"
ExecStart=/home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn --workers 3 --bind localhost:8080 -m 007 wsgi:app

[Install]
WantedBy=multi-user.target

#

yeah but

#

doesnt say that

#

it's telling me to put the location of the .py file, and the path variable

#

and then using gunicorn

#

which would be stored in the bin file

gusty willow
#

does expadmin/www-data actually have permissions to that environment?

sand thistle
#

it may not

gusty willow
#

ls -lash /home/expadmin/ExpFlask/management_dashboard

#

my guess is, you are going to be seeing expadmin:expadmin on ownership

sand thistle
#

so i accidently cd into the directory

#

but

#

everything has root except

#

the venv folder

gusty willow
#

so it's all root:root?

sand thistle
#

which is where gunicorn is actually stored

#

expadmin:expadmin for that one

#

as well as

#

expadmin for ..

gusty willow
#

ok, so there are no folders that the "www-data" group actually has permissions then

#

so that is your problem

#

generally, i don't like to run sites out of /home...

#

i tend to use /var/www or something of that nature

sand thistle
#

i see

#

this is my first deployment

#

so then I need to switch permissions for which folders?

#

chmod

gusty willow
#

in your case, you first need to make sure that your user is a part of that group sudo usermod -aG www-data expadmin && newgrp www-data

#

then, make sure that the folders that are serving all of that have the correct permissions sudo chown -R expadmin:www-data /home/expadmin/ExpFlask

sand thistle
#

erm I'm missing an operand

gusty willow
#

and for good measure, i would set the permissions like so sudo chmod -R 0770 /home/expadmin/ExpFlask

#

if the group www-data doesn't exist, you may get an error ... in which case sudo groupadd www-data

#

but that is easy to find if you just grep www-data /etc/group

sand thistle
#

www-data:x:33:expadmin

gusty willow
#

ok, so the group exists, that is good, but yeah, do the chown/chmod stuff and then try to run your .service file

sand thistle
#

then, make sure that the folders that are serving all of that have the correct permissions sudo chown -R expadmin:www-data /home/expadmin/ExpFlask
@gusty willow am i supposed to specify the folders here, after ExpFlask

gusty willow
#

the -R means recursive, so anything under that will get those permissions

sand thistle
#

ah sry

#

discord messed up the syntax a bit

#

yeah so

#

i did all that, and double check permissions

#

different error

#

now it just wont start

gusty willow
#

what error

sand thistle
#

Main process exited, code=exited, status=203/EXEC

#

before it couldnt locate the file

gusty willow
#

right, so it ran, but couldn't find anything, now it can but it's failing for whatever reason ... we are getting closer ๐Ÿ˜„

#

can you add --log-level debug to your .service file

#

and try again

#

maybe we can get more info

#

also, do you actually have gunicorn installed the venv? not just the global?

sand thistle
#

it's in my requirements.txt

#

and gunicorn is located in the venv folder as well

#

under bin

gusty willow
#

just making sure ๐Ÿ˜„

sand thistle
#

can you add --log-level debug to your .service file
@gusty willow this would be under the service portion of the systemd file

gusty willow
#

but yeah, in your ExecStart line, add --log-level debug to the end or wherever you want to add that particular argument

#

again, docker solves a lot of these particular issues lolol

sand thistle
#

hmm

#

I added this after the end

#

--log-level debug

#

--workers 3 --bind localhost:8080 -m 007 wsgi:app --log-level debug

#

now its the same error as prior

#

expDashboard.service: Failed to execute command: No such file or directory

#

we just fixed this error

gusty willow
#

maybe it needs to go before the -m

sand thistle
#

I may have entered it incorrectly

gusty willow
#

i'm checking the syntax real quick ๐Ÿ˜„

#

yeah, it's just --log-level LEVEL

#

but yes, it does need to be before your wsgi:app declaration

#

my bad

#

do you have your code somewhere in git that I can see it all in totality

sand thistle
#

I hate everything

#

do you mean my systemd file, do I have that on github

#

no

#

ExecStart=/home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn --log-level LEVEL --workers 3 --bind localhost:8080 -m 007 wsgi:app

gusty willow
#

you need to do --log-level debug ๐Ÿ˜„

sand thistle
#

whoops

gusty willow
#

and also, looks like you have too many spaces after the level

sand thistle
#

same error

gusty willow
#

and what happens if you do newgrp www-data && /home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn -h just from your terminal

sand thistle
#

nothing happens

gusty willow
#

what about just running /home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn -h

#

my suggestion here is to probably pip install --user gunicorn and then adjusting your .service file to call /usr/local/bin/gunicorn rather than from inside the venv though

sand thistle
#

so

#

umm

#

this

#

/home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn -h

#

yields

#

no such file

gusty willow
#

ooo, ok, so there are problems lolol

sand thistle
#

when i cd into it though

#

it exists

gusty willow
#

so, truely, if you want my honest opinion, run a container, it'll solve all of this

sand thistle
#

i was advised heroku as well

gusty willow
#

heroku would work, but it's a bandaid for one off things

#

to learn how to really do deployments, i wouldn't learn that

#

do you have docker installed?

sand thistle
#

i dont

gusty willow
#

i can walk you through this pretty easily after you get it installed

sand thistle
#

youre suggesting i use a docker container in my droplet?

gusty willow
#

yeah, i do it all the time ๐Ÿ˜„

#

there are docker base images too

#

for droplets

#

that might be easier

#

idk if DO has a direct "run this container" option, but if they do, that would be best lol

sand thistle
#

im installing it

gusty willow
#

and does your code exist on that droplet? i assume it does

sand thistle
#

yes

#

it does

#

honestly

#

I need to eat

gusty willow
#

๐Ÿ‘

#

i'm writing you a quick dockerfile to work with

sand thistle
#

im very frustrated

lost night
#

When creating a python script which needs an api key, what's the best way to provide it to the script?

#

Python config file, JSON config file, enviroment variable or command line argument?

covert kindle
#

I've never used a mac, but im curious if the default terminal is bash or has bash-like commands/keystrokes.

Also, I assume vim is pre-installed?

sand thistle
#

default terminal is bash @covert kindle

#

i use nano so I can't say, but I believe vim is preinstalled

covert kindle
#

Awesome, thanks!

gusty willow
#

@lost night I would argue that none of them are preferred over a dedicated secret store

#

@lost night but of those four, envvars or arguments make sure you aren't going to git commit secrets

#

@covert kindle I can't remember if the preinstall is vi or vim, but it's one of those lol

sand thistle
#

it's not vi

covert kindle
#

@sand thistle

Dockers are an awesome tool, I use them daily for development and for random testing. They come in super handy since I use windows but most of my tools are unix. I even used them to replace an old FreeNAS build.

The docker classroom is great for getting started. Definitely a tool that is worth learning, especially with its many uses.

Play with Docker Classroom https://training.play-with-docker.com/

Learn docker through online trainings in training.play-with-docker.com

#

I used them for deploying a Django/PostgreSQL/Redis/NGINX/gunicorn/Daphne project. Worked great.

tawdry needle
#

what's daphne?

lost night
#

thanks @gusty willow

lone linden
#

Asgi webserver iirc

gentle apex
#

if im wanting to run a app that uses python and js do i need to build my docker file from base image or can i use the python image and install node in there?

tawny temple
#

Both will work

#

Using a base image is just more convenient

#

As in, using a node or python image as a base is more convenient than creating one yourself

gentle apex
#

im just not comfortable with base images cuz idk how to specify python version. seems like the only thing i can ensure is that the python is 3 not 2 lol

tawny temple
#

What do you mean by base image

gentle apex
#

like from os

#

cuz when u run install python3 u kinda cant specify the .x at least from what i know

tawny temple
#

It depends on the OS I suppose. Usually you don't use a rolling release style OS for images, so you're "stuck" with what the OS package index has, unless you want to build and install python from source or use a tool like pyenv. That's why using a Python image as a base is better than a bare OS image.

gentle apex
#

i was gonna start with bare os and install python and node

#

but it seems idk how to specify the python version

#

like u can only do install python3 not sth like install python3.8 or sth

tawny temple
#

Why not pick one as the base and install the other

gentle apex
#

yeah i think that prob much easier

#

but just curious how do u specify python version tho

tawny temple
#

See the message I sent 2 messages ago

gentle apex
#

oh

#

ic

#

i think ima stick to using the python as base from now on...

tawny temple
#

The same thing applies to Node, so you could run into the problem anyway if you need a newer or older version than the OS has.

#

Unless you find an image with both Python and Node that has the versions you need.

#

Granted, I'm not really able to imagine how a single app could rely on both Python and Node. Are you sure you're not trying to stuff 2 separate apps into one container?

gentle apex
#

its some flask thing

#

that im trying out

sand thistle
#

can you stack commands in docker

#

like can you do a login and a run

tawny temple
#

log into what?

#

And run what

#

You can run a shell in the container, which will let you execute arbitrary commands, if that answers your question.

gusty willow
#

@tawny temple I was working with him, he meant logging in with docker login and then running the container

tawny temple
#

I see. I'm not sure why you'd need a login before a run, but I suppose you could do it with basic shell syntax docker login && docker run ...

gusty willow
#

The login only if you are pulling from a private repo

#

Or a cloud provider repo

tawny temple
#

Oh right, run will pull if it doesn't exist

snow cargo
#

There are so many CI/CD products and I can't decide which one to use

#

Jenkins seems to be both the most hated and loved...?

tranquil creek
#

I don't get the hate for Jenkins. It does what I need. The interface looks dated, but the shit works just fine.

safe gull
#

how do i use a virtualbox
i need to use linux for python, as i will compete in hack the box (an online place to hack people's vm)

wooden ibex
#

superuser, you wouldn't use the same container, you would have python container and JS container

warm pollen
#

I'm looking for a website where I can upload a simple install script, so I can curl | bash it, but I'd like the url to be pretty small, and github raw urls aren't short at all, do you have any site to suggest?

obtuse rapids
oak path
#

I have the github student pack,I'm trying to sign up to aws but I don't have a credit card, what do I do?

tawdry needle
#

@warm pollen could just use a url shortener

#

or spin up a basic FTP/HTTP server

warm pollen
#

I'm 'ot sure about how long pastebin stores pastes

#

I guess an url shortener could work

#

It is intended to be public, so I was wondering if they were a service dedicated to that

wooden ibex
#

github?

warm pollen
#

They have really long urls

tawdry needle
#

im surprised there isnt some "file hosting for hackers" site with short urls

gusty willow
#

this is a thing, but it's on tor lol

tawdry needle
#

hmm

#

im thinking of just a dumb http/ftp server where you can upload up to say 10 mb of files

#

with some up/down bandwidth cap

#

and short cute URLs

#

nothin like that exists currently?

wooden ibex
#

people would use it for bad stuff most likely

#

Akarys, use url short to point to github raw?

warm pollen
#

Yeah, I guess I'll do that

tawdry needle
#

@wooden ibex you can use github for bad stuff too though

#

i heard thats why firefox send was pulled, but i dont see how this isnt an issue for literally any other site that hosts user provided data

heavy knot
#

anyone here with an experience of scraping pdfs? i want to parse pdf tables with very odd formatting, there are 2 types of column in one table (!) each of different width, multirow cells and also multicolumn cells, on top of that tables span across multiple pages
any ideas how to approach this task?

tawdry needle
heavy knot
#

@tawdry needle thanks, i'll give it a try, I tried tabula-py already and didn't have much luck with it

#

well, it works better than python version, still doesn't store multicells correctly like i want a value to be stored in each subcell

tawdry needle
#

ah, too bad

#

pdfminer is the python pdf parsing library that i know of

heavy knot
#

as per its github page, it is not actively maintained

#

now i have one more option to try ๐Ÿ™‚

tawny temple
#

These PDF libs may already work this way, but consider exporting the pages to images and then using a library that can extract tables from images rather than PDFs. May be worth a shot.

heavy knot
#

@tawny temple thanks for suggestion, that's an interesting approach, i didn't thought about it

#

my main problem with pdf libs is that if there are multicells as well as empty cells on a single line it is hard to distinguish between NULL values from them

#

maybe those OCR libs will be able to deal with it more effectively

bold grove
fathom silo
#

if python IDLE and Atom had a baby what would it be?

tawdry needle
#

@fathom silo vs code?

#

since vs code has a python language server and repl functionality built in

#

sublime text supports LSP and has sublime REPL but its honestly not as good as vs code

#

and there are always more "full featured" IDEs like pycharm

fathom silo
#

can you run code directly off of it

tawdry needle
#

yeah, i dont use vs code myself but there's some setting or keyboard shortcut to run a whole file or a line from a file

fathom silo
#

that is literally all I wanted thanks

tawdry needle
#

ive been thinking of switching to vsc now that their python support is better and sublime text plugin development is kind of in a rough state

#

rip sublime text community ๐Ÿ˜ฆ

fathom silo
#

it's 70 bucks now right?

tawdry needle
#

its always been $70

fathom silo
#

oh

tawdry needle
#

the problem now is that the plugin dev community is nowhere near as active as it used to be

#

so its ripoffs (atom and vs code) are starting to surpass it

fathom silo
#

did they migrate to somewhere else?

#

oh

tawdry needle
#

no idea where they went, i think people just switched away from sublime

#

the devs went silent for basically years before 3.0 was out of beta

#

sublime is a rock solid stable editor still and the devs earned by $70 many times over

#

but its sad to see the plugin ecosystem starting to wither

#

lots of years-old unmaintained plugins now

#

i think they should just open source sublime or something, idk

fathom silo
#

you know if you can run code directly from atom without any plugin?

tawdry needle
#

im not sure, atom might have some kind of repl functionality built in but i wouldnt know

fathom silo
#

just googled there's an extension with 1.8m installs that does exactly that

#

"script"

tawdry needle
#

is an extension not the same thing as a plugin?

fathom silo
#

guess so

#

package is the name they give it

#

fucking hell 3m installs

rustic coyote
#

vs code is just adding features at a relentless pace and they place special emphasis on python

tawdry needle
#

yeah vs code is very heavily funded by msoft right now

#

putting a lot of work into typescript too i think

#

and the language server protocol itself

fathom silo
#

why do they do it tho

#

I would understand if it wasn't available to linux but how do they profit off of it?

oak path
#

i'm trying to host my discord bot on azure, I SSHed and typed python <filename> it runs fine but when I close my terminal it kills the process, why?

tawdry needle
#

@fathom silo i think its basically a big PR move

#

embrace extend extinguish except they arent necessarily going to extinguish

#

vs code is a gateway into a whole new "dev friendly" microsoft ecosystem

fathom silo
#

oh I see

tawdry needle
#

im not sure what else ms gets out of vs code specifically

#

but i know that supporting e.g. typescript helps ensure that typescript is good enough and usable for their purposes

jolly iris
#

@oak path if you quit your ssh session your process receives a hangup signal from the server resulting in killing your process

oak path
#

oh

#

how do I avoid that?

jolly iris
#

You could try nohup

oak path
#

what is that?

jolly iris
#

something like nohup <process> &

oak path
#

okay

tawdry needle
#

i use tmux

jolly iris
#

nohup - run a command immune to hangups, with output to a non-tty

#

@tawdry needle with detaching the session after running the process? this will also work

tawdry needle
#

yep exactly

#

i also like having multiplexing anyway

oak path
#

if I want to run a jar file on the side what will be the port?

wooden ibex
#

salt rock lamp, Azure

#

VSCode is gateway drug to Azure

#

also, Microsoft needed editor for their Python/Java programmers

#

and Rust, can't forget that

tawdry needle
#

if that isn't an indictment of how horrible visual studio is, then i dont know what is

wooden ibex
#

Nothing

#

Visual Studio has always been designed for .Net programming

#

it has plugins for other stuff but Python/Rust/Javascript have always leaned towards more lightweight editors and Microsoft realized providing said lightweight editors was key to winning them so they did it

#

We do .Net at work, Once you get over learning curve of Visual Studio, it's ultra powerful (like PyCharm)

tawdry needle
#

fair enough

#

isnt it also the main C/C++ compiler and IDE on windows?

wooden ibex
#

for C/C++ it works well

#

for Python/Javascript/Typescript, I believe VSCode is most popular on Windows

tawdry needle
#

well for python pycharm is still quite popular. i dont know anyone who uses visual studio for python

wooden ibex
#

I mean, there is a plugin for it.....

ebon spire
#

is this the channel for conda related question too?

tawdry needle
#

sure, conda is on topic here

ebon spire
#

In conda how can I remove only packages installed from a certain channel?

#

the other one is how can I clone the base environment as it like when conda is newly installed which I mean not to include those new added packages (which might also mean the rev 0 in conda list -r)

gusty willow
#

@oak path no idea what you are talking about

oak path
#

Np

gusty willow
#

@oak path no, I mean what are you talking about lol. Jar files are java executables that would be configured in their own way, so there is no way we could tell you what port that app runs on. And how does that relate to python anyway? I'm just generally confused.

oak path
#

I just wanted to run 2 scripts

#

It works now

#

Sorry, I wasn't being clear

tawdry needle
#

@ebon spire the last column of conda list output is the "Channel" column. you can use --show-channel-urls to get more detail in that column, and of course you can use --json to pass it to another program like JQ

tawdry needle
#
read -r -d '' jq_filter_prog <<'JQ'
del(
    .[]
    | select( .channel != "pypi" ))
  | map(.name)
  | "conda uninstall " + join(" ")
JQ
read -r -d '' uninstall_command < <(conda list --json | jq -r "$jq_filter_prog ")
echo "$uninstall_command"

@ebon spire if you want to get extra-slick using jq ๐Ÿ˜›

#

as for the 2nd question, im not sure. are you trying to "reset" the base env?

mystic flint
#

hello, assuming i am running a script on a hoster, and this script is downloading files to its root directory that i need to keep - the hoster, however, resets daily. what is the simplest and best free solution to storing these files?

#

i know about aws s3 but that doesnt seem very simple to me

modern stump
#

Unfortunately, that might be the solution you have to use if you want to work with that platform... S3 gives you 5GB for twelve months for free, if that makes it any better. If your files are small enough/formatted consistently, you can store them in a database

#

What platform are you using?

tawdry needle
#

s3 is pretty straightforward for a single user. just make sure it's set to private, not public

#

you can also use backblaze b2, i have good experiences with that company

#

or google cloud store, or azure blob store, etc.

modern stump
true vapor
#

Does anyone have a decent production ready Django Docker setup I can look at?
preferably a relatively simple one with no bells or whistles

tawdry needle
#

i dont have one, but you'd typically be looking at just python3 + django + gunicorn + your app, right?

wooden ibex
#

Does Django not have prebuilt container image?

#

Nope

true vapor
#

There's a lot of random small but important considerations

tawdry needle
#
# Your settings should already be inside ./app/, or use the Docker --env--file option
# Include Django and Gunicorn in requirements.txt

FROM python:3

RUN mkdir /app
RUN python -m venv /venv
COPY ./ /app/

WORKDIR /app
RUN /venv/bin/pip install -r requirements.txt
RUN /venv/bin/pip install .

EXPOSE 8000
CMD ["gunicorn", "--bind", ":8000", "--workers", "3", "myapp.wsgi:application"]

this seems like the absolue bare minimum django+gunicorn setup, assuming your python app itself is sane

#

if you want to be really slick, you can (should) use pip-compile to generate the requirements.txt file from requirements.in

wooden ibex
#

Yea that link is awful, when you are install deps shit is going sideways

tawdry needle
#

yeah, installing deps by compiling from source... yikes

#

oh its not from source but still

#

@wooden ibex aren't there times when you just need to use the package manager in docker?

#

maybe this is just them wanting to keep the container smaller by using FROM python:3.7-slim and then installing only what they need

wooden ibex
#

Which is dumb

tawdry needle
#

im more alarmed by all the hard-coded env stuff in the dockerfile

wooden ibex
#

Thatโ€™s standard developer dumb which SRE will come in and beat with a stick

tawdry needle
#

how do you feel about changing USER inside the dockerfile? i just added that to my snippet but i dont typically see it

#

so i wonder if there's some reason people dont normally use it

#

RUN groupadd -r ${APP_USER} && useradd --no-log-init -r -g ${APP_USER} ${APP_USER} seems weird and too "manual" for docker

true vapor
#

Yea that link is awful, when you are install deps shit is going sideways
That is very sad news

#

That looks like a much better guide

wooden ibex
#

salt rock lamp, it's considering best practice because running as root

tawdry needle
#

thats what i figured. so you need to manually create the user inside the dockerfile if your image doesn't do it for you?

wooden ibex
#

@tawdry needle NO

tawdry needle
#

oh lol

true vapor
#

This is the suggested dockerfile from the DO article - I don't honestly understand it, but that RUN set seems funky

ADD django-polls/requirements.txt /app/requirements.txt

RUN set -ex \
    && apk add --no-cache --virtual .build-deps postgresql-dev build-base \
    && python -m venv /env \
    && /env/bin/pip install --upgrade pip \
    && /env/bin/pip install --no-cache-dir -r /app/requirements.txt \
    && runDeps="$(scanelf --needed --nobanner --recursive /env \
        | awk '{ gsub(/,/, "\nso:", $2); print "so:" $2 }' \
        | sort -u \
        | xargs -r apk info --installed \
        | sort -u)" \
    && apk add --virtual rundeps $runDeps \
    && apk del .build-deps

ADD django-polls /app
WORKDIR /app

ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

EXPOSE 8000```
tawdry needle
#

do we hate alpine too? @wooden ibex

#

set -ex is just a bash thing to stop it from continuing if one line errors out

wooden ibex
tawdry needle
#

dont install postgres into your container tho...

#

that much i know not to do

#

and this is bad/weird, just use /env/bin/python and /env/bin/pip explicitly

ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
wooden ibex
#

what salt rock lamp posted looks good

true vapor
#

the DO article?

wooden ibex
#

no

true vapor
#

just the tiny dockerfile?

wooden ibex
#
# Your settings should already be inside ./app/, or use the Docker --env--file option
# Include Django and Gunicorn in requirements.txt

FROM python:3

WORKDIR /app
COPY . .
RUN pip install -r requirements.txt


EXPOSE 8000
CMD ["gunicorn", "--bind", ":8000", "--workers", "3", "myapp.wsgi:application"]```
#

from Salt rock lamp

true vapor
#

yeah, sure. It misses out some stuff though, mainly migrations and staticfiles which I have no idea how to handle in production. Although that isn't in Docker's domain I don't think

wooden ibex
#

I edited it

#

don't do venv in docker, it's pointless, that's whole reason for docker

tawdry needle
#

@wooden ibex some python people feel strongly about using it anyway

#

at least, #python on freenode does

wooden ibex
#

I mean, it's not end of the world but whatever

#

they probably think Alpine is good idea too

true vapor
#

isn't alpine just sort of objectively bad?

wooden ibex
tawdry needle
#

this is good to know

#

Why? Most Linux distributions use the GNU version (glibc) of the standard C library that is required by pretty much every C program, including Python. But Alpine Linux uses musl, those binary wheels are compiled against glibc, and therefore Alpine disabled Linux wheel support.
ew

#

suckless, amirite?

wooden ibex
#

it's all to save DISK SPACE

#

something few people have in short supply

tawdry needle
#

nah bro it sucks less!

#

i too recompile my entire window manager from scratch when i change settings, because config files are bloat

wooden ibex
#

I just use Win10 and problem solved lemon_fingerguns_shades

#

BTW Charlie, I edited salt rock lamp docker file

#

see if that works

true vapor
#

It's not gonna work right now - I don't have a database set up and running locally

#

but that's what I currently have in my Dockerfile

tawdry needle
#

this is what docker compose is good for right?

#

1 dockerfile for database, 1 dockerfile for django app

wooden ibex
#

yep

#

ignoring database in docker

warm pollen
#

You'd usually pull the database container from dockerhub though

tawdry needle
#

nginx supports binary streams now too right? so you can put them both behind nginx and have a nice time

warm pollen
#

You can even put nginx inside a container lemon_fingerguns_shades

#

It works quite well actually

wooden ibex
#

nah, I would just have django app make a call to database over internal docker network

tawdry needle
#

i mean if you also need to expose the database outside

true vapor
#

Right now I have an attrocioius setup on a OVH VPS using some monstrosity dockerfile and some monstrosity DB running in a dockerfile. I'm trying to get it set up in a way that I can just plonk it on GCP and use a cloud SQL db

tawdry needle
#

@warm pollen im actually curious about how that works, do you have any setup examples?

warm pollen
#

About nginx?

#

We did that for the jam, hang on

wooden ibex
#

salt rock lamp, you can configure docker to have internal only networks

warm pollen
wooden ibex
#
      - ./config/nginx:/etc/nginx/conf.d
      - ./static:/static``` ![lemonshake](https://cdn.discordapp.com/emojis/436911467598577664.webp?size=128 "lemonshake")
#

BTW, you can have docker compose use docker files for build

warm pollen
#

Unh?

#

Wdym?

#

(the bind mounted config is because time is flyin' haha)

wooden ibex
#
  drblistener:
    build:
        context: .
        dockerfile: discord_listener.dockerfile```
#
FROM python:3.6-slim-buster

# If you prefer miniconda:
#FROM continuumio/miniconda3

WORKDIR /app
COPY Modules/ Modules/
COPY parameters/parameters.json parameters/
COPY requirements.txt .
COPY discord_listener.py .

# Using pip:
RUN python3 -m pip install -r requirements.txt
CMD ["python3", "./discord_listener.py"]```
warm pollen
#

Riiight

wooden ibex
#

No I get while you are using volumes, I just not a fan of them for stuff that should be checked into git, shoved into container

#

volumes should only hold data you can't rebuild (like database files)

warm pollen
#

Yup I agree with that actually

wooden ibex
#

I wish Jams would let SREs participate

gusty willow
#

code jams?

warm pollen
#

I usually put my COPYs last so I can just rebuild the container every time

#

SRE?

wooden ibex
#

Site reliability engineers

#

People like j4ng5y and I and others who would be cranky about docker files

gusty willow
#

๐Ÿ˜„

true vapor
#

You could participate. Of course it's jam dependant, SRE stuff isn't at all relevant for Kivy desktop apps, but having someone handling all the SRE stuff for Django would probably have been useful in the last jam

wooden ibex
#

Charlie we would occupy a slot going to developer

true vapor
#

There's 5 people in a team, usually up to 3 are inactive

#

so - most teams wouldn't complain too much

#

you could always mention it in meta - suggesting that some amount of emphasis be put on stuff outside of code

warm pollen
#

Yeah, I personally wouldn't have minded

gusty willow
#

Code, though obviously important, is usually only like 40% of the game lol

#

making it run is arguably more important lol

#

chicken vs egg though

true vapor
#

God I hate this devops side of things. I think this Dockerfile works, but now I need to write a damn docker-compose file for the database, and make sure my app is getting the database creds correctly

#

and

#

I just don't care

#

I know the code works

#

I wrote it

#

but I have to care :(

gusty willow
#

thissaid every developer in the era of devops lol

#

shift left they said, it'll be great they said

#

but really, i spent a good chuck of my time at my last gig just making dev lives easier in that regard (i ultimately ended up suggesting and deploying skaffold bundled with k3s with some template repos that had basic manifests for common tasks)

wooden ibex
#

there also isn't a need for every team to have SRE, most of time once we setup docker and github actions, it's done

gusty willow
#

yeah, we had 300+ engineers and a SRE team of 5 and we managed just fine lol

#

gitlab here, but same thread

mystic flint
#

@modern stump yea heroku, ive been thinking that i can use dropbox but i dont know what the limits there are, amazon seems to have a pretty complicated setup and apparently needs verification via credit card?

gusty willow
#

@mystic flint not sure what you are referring to, but using dropbox as "app storage" doesn't sound like a good idea lol

mystic flint
#

its just for a bunch of small files (mp3s) that are like 2-5 mb in total

gusty willow
#

if you don't care about speed, then yeah, probably ok. But don't expect to use that on a resume ๐Ÿ˜„

devout temple
#

how does one set the python path? i'm setting up an espressif project in visual studio and it complains about how it can't recognize pip/python in my environment

#

this didn't work```

PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin/${PATH}"
export PATH ```

gusty willow
#

@devout temple first of all, you set the path incorrectly ๐Ÿ˜„

#

assuming that is the correct filepath to your python binary: should be export PATH=/Library/Frameworks/Python.framework/Versions/3.8/bin:${PATH}, you missed the :

#

most of the time though, i just suggest people use brew on a mac

devout temple
#

right...but does it set PYTHONPATH?

#

have you used visual studio? I changed the path for python but it still seems to be using an older path that's inaccessible

devout temple
#

okay so I put Library/Frameworks/Python.framework/Versions/3.8/bin/python3 as the python absolute path but it still complains `Python or pip have not been found in your environment.

`

lament zephyr
#

ayo how do i make a script run periodically

true vapor
#

Cron if you're on Linux

thorny narwhal
#

hello all, i don't really know in which category i should put this question however : Does anyone know what extension, library i can use in python which will control notifications appearing in windows sidebar?

#

exactly click on it when it appears in bottom right corner

#

dont really know if autopygui has this possibility

gusty willow
thorny narwhal
#

i posted this question in general, in 30secs i get answer. thanks!

wooden ibex
#

One of cloud provider

jaunty spear
#

I'm pretty new to python, and i was wondering some things about using Zoom API and stuff message me if you want to help

midnight atlas
#

can someone plz help me?

#

@heavy knot Can you help me? auto-py-to-exe

heavy knot
#

I've never used that before

unkempt tendon
#

Hello, about making automation scripts is right channel?

copper meadow
#

@heavy knot Can you help me? auto-py-to-exe
@midnight atlas I can

warm pollen
#

Hello, about making automation scripts is right channel?
@unkempt tendon sure

rough marlin
#

@pearl pawn this might be a better channel actually, it'll be quieter.

#

I'm also not sure how vim and neovim compare

#

I've been working on learning vim for the past few days

pearl pawn
#

@rough marlin thanks

#

Does anyone know how to fix indentation problem with comments?

rough marlin
#

are you pasting from somewhere?

gusty willow
#

I've seen that actually in neovim. What plugins (if any) are you using@pearl pawn

pearl pawn
#

@gusty willow

gusty willow
#

Fwiw, you don't have to use plug anymore

pearl pawn
#

That's true but it doesn't make a difference at my level of vim usage

shut brook
#

Anyone here do python development from an iPad?

lime sable
#

do i need to do something extra for pycharm to detect a module in a subdirectory?

#

it keeps saying "can't find module" even with the correct source directories set

wooden ibex
#

Verxion, try something like repl.it but it's going to subpar experience.

midnight atlas
#

@midnight atlas I can
@copper meadow no need, it was my antivirus that was blocking it โ›”

copper meadow
#

yea, it was the same case with me too once @midnight atlas

midnight atlas
#

๐Ÿ‘

drowsy lava
#

someone help me

#

I'm not sure what any of this means and I wanna get rid of it

midnight atlas
#

the argument "master" is not a argument in 'git rev-prase'@drowsy lava

drowsy lava
#

oh didn't know that existed let me go thru it

#

probably should've

midnight atlas
#

๐Ÿ‘

drowsy lava
#

oh wait no not the one I'm highlighting

#

I meant everything.

midnight atlas
#

oh

drowsy lava
#

The 5000 changes and a repo branch that doesn't exist I can't remove them

midnight atlas
#

try ping a helper

drowsy lava
#

huh alright

#

I'm not sure how this happened

#

I just enabled the git extension and there came out

midnight atlas
#

holi ****

drowsy lava
#

what

#

oh i see what you were saying lol

midnight atlas
#

that it a lot of files

drowsy lava
#

yes and I have no idea what is happening

midnight atlas
#

๐Ÿ˜ฉ

drowsy lava
#

nice pfp though

midnight atlas
#

have to go

mighty venture
#

I am getting a very frustrating error with running docker:

duck@system:~$ docker-machine create -d virtualbox testdriven-dev
Running pre-create checks...
Error with pre-create check: "VBoxManage not found. Make sure VirtualBox is installed and VBoxManage is in the path"

I have already installed virtualbox and vboxmanage is on the PATH, I can run it on my own but docker can't.
I have already looked up docker forum and stackoverflow with no solution

duck@system:~$ which VBoxManage
/usr/bin/VBoxManage
duck@system:~$ which docker
/usr/bin/docker 

Status: Solved

topaz sable
#

does git rebase <branch> put commits from the currently checked out branch onto the one specified in <>?

#

and how would I rebase from a specific hash?

warm pollen
#

You can specify a commit/commit range/branch after the target

signal crest
#

Can someone ELI5 what Jupyter notebooks is and what it's used for? It looks sort of like a REPL session that you can also save to file and stuff. is that accurate? and if so, what's the use for it?

gusty willow
#

@signal crest Jupiter notebooks are generally used for scientific documentation and data science documentation, but I find them useful in teaching python to junior devs as well. As far as what it does, notebooks allow you to organise your code in a document like fashion, write documents just like you would in Word, but you are also able to write live code within the document. It's kind of hard to explain all in one go lol.

#

To be more ELI5 though, is a program to run code and write documents all in one.

signal crest
#

Hm, I don't think I've seen it being able to embed word-like text before. that does seem kind of useful.

gusty willow
#

Not exactly word like, but the formula parsing gives it more of that feel

#

And is a whole lot easier than TeX lol

signal crest
#

so basically it's purpose is mostly presentation of code rather than creation? that would explain my confusion because I always thought it was meant to help write code somehow

gusty willow
#

You can write code with it alone, but it's not as easy as a real editor or an ide

#

But if you need to write a technical paper, it's perfect lol

#

I do like some of the things that iPython brings though, but you don't have to run notebooks to get it

pearl pawn
#

Is anyone active?

#

@gusty willow I'm ating you because you helped me the other time my bad. But do you know anything about tcp proxies. I need help understanding why my code works

gusty willow
#

Lol, usually the question is the other way around

pearl pawn
#

lol ik

#

but its weird

#

ill show u

gusty willow
#

As far as your question, yes, I know and understand proxies lol

pearl pawn
#

@gusty willow The website is sending Data to my proxy and all my proxy is doing is forwarding it to the client yet my client gets the entire content of the page while my proxy gets the permanant move address error

#

yet it works

#

idk y

gusty willow
#

Problem is your proxy is seeing the initial request, but not following through with it. The client received the same message, but followed the redirect, and therefore, actually got the data back. @pearl pawn

pearl pawn
#

Huh, ok I'll figure this redirect problem out then. Thanks bro!

#

Could it be because its http and not https?

thorn berry
#

hello everyone
i currently have a command line tool
and i want to make sure that a command doesn't work if a user hasn't run a command before
for example
package_name supercharge
and only if they run that command before, this should work
package_name superpower

sly sleet
#

save it in a file

#

@thorn berry

thorn berry
#

@sly sleet where would be a save protected location? I need to store a bool variable so.. is there any easier way/

sly sleet
#

hm

#

there's not much you can do to prevent them from reading it

#

chmod/chown could help

#

but they could always use root

thorn berry
#

true... though, how can i store it into some kind of system variable?

#

like a quick access variable?

warm pollen
#

Writing somewhere in /etc will be root-protected

mortal panther
#

hi guys! I could not find anything like this online, so here is a class that allows you to run any command with input/output processed by python. supports commands that require user input:
https://github.com/vemel/awscliv2/blob/master/awscliv2/interactive_process.py

InteractiveProcess(["ls"]).run() # just run and exit
InteractiveProcess(["aws", "configure"]).run() # this one uses user input

Note that it does not use shell=True, so should be secure. Happy keylogging!

thorn berry
#

Check this out, it's really amazing!

mortal panther
#

@thorn berry nice! what does it offer compared to pip?

thorn berry
#

no, its actually not a competitor to pip

#

you can install it using pip

#

@mortal panther it is for installing applications and packages into linux

mortal panther
#

omg, I see ๐Ÿ™‚ cleaner README is a must then

thorn berry
#

thanks for the feedback @mortal panther i will work on that

#

I've been spending 7-8 hours for the past 2-3 days trying to improve it, haven't gotten around to the docs and stuff lol

#

in fact, unlike other package managers, i am planning to add multi-threaded performance, so you can install packages at the same time

mortal panther
#

i believe apt uses multithreading

thorn berry
#

that i think it true however, you cannot run multiple instances of apt

#

anyway to install multiple packages of apt I will need to make sure that I write some complex code, to increase the speed

#

lol though i am kinda the only one working on this right now, so its going to be very hard for me to do so...

true vapor
#

how do I point a Django docker image/container to a database running in another docker container? The Django one needs to access it at both build and run time

wooden ibex
#

Build a network

true vapor
#

the fact it's django probably doesn't matter. Just how can I set things up so that one docker thing can access a database running in a different docker thing

wooden ibex
#

Why would access be needed during build?

true vapor
#

to run database migrations

#

this is only for local stuff though

wooden ibex
#

That should be separate container

#

Depending on your needs you can rely on bridge network since they will be separate ports

#

Iโ€™m on mobile so I canโ€™t look at docker setup

true vapor
#

I think I'm just gonna use a VPS on the internet for testing, which is all I need right now

#

and I know that that works and how that works

wooden ibex
#

Is this a public repo?

true vapor
#

no, and annoyingly I can't make it public either

wooden ibex
#

Ok, then

true vapor
#

This is a pretty temporary thing, I just want it done with

wooden ibex
#

Well carry on, I was just going to peek and see

true vapor
#
# Your settings should already be inside ./app/, or use the Docker --env--file option
# Include Django and Gunicorn in requirements.txt

FROM python:3

WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
ENV DB_NAME=db \
    DB_USER=docker \
    DB_PASSWORD=docker \
    DB_HOST="goddammit.just.work" \
    DB_PORT=5432

RUN python manage.py collectstatic --noinput
RUN python manage.py migrate --noinput
EXPOSE 8000
CMD ["gunicorn", "--bind", ":8000", "--workers", "3", "brac.wsgi:application"]
version: '3.1'

services:
  cheapestpintin_db:
    image: postgres
    restart: always
    environment:
      - POSTGRES_PASSWORD=docker
      - POSTGRES_USER=docker
      - POSTGRES_DB=db
#      - PGDATA=/var/lib/postgresql/data/pgdata
    container_name: cheapestpintin_db
    ports:
      - 5432:5432
#    volumes:
#    - ./volume/dbx:/var/lib/postgresql/data

That's the dockerfile for the app and the compose file I'm running the database with. Not all that interesting

#

the actual app is just generic boring CRUD django

gusty willow
#

lolol @ goddamnit.just.work

#

so ... is the django app included in the compose file?

true vapor
#

no, the compose file is literally just the database - because in production the database will be on GCP

#

this is just me trying to figure out how to get the app ready for that

topaz aspen
#

what approach do people take to ensure that they're up to date with master? Files like setup.cfg and stuff like that ๐Ÿค” is there a hook for that? Or do you just diff against master

gusty willow
#

@true vapor then just add it to the compose file, you don't have to leave it that way when you are ready for deployment, but that simplifies intra-container networking greatly

#

@topaz aspen that is why i use branches, so I don't have to worry about it. I only deal with sync problems when I'm read to merge

topaz aspen
#

@gusty willow i have branches hang around for a while sometimes :/

true vapor
#

I'll have a look at doing that - I treid a while ago and messed it up. But yeah, you're right, I should just figure it out

gusty willow
#

right ... so do i, but when i'm ready, i'll just merge in the final destination and deal with problems

true vapor
#

I kinda forgot it was a thing

topaz aspen
#

sometimes there are changes on master i want to bring in though, i'll just diff it's fine

gusty willow
#

@true vapor the reason you are probably having so many problems is the fact that one is running as a true container, and the other in a compose ... compose creates a new network isolated from the rest of the docker environment, so without a little hacking, it's hard to make the two talk ... so you need to do either 100% compose, or 100% not compose lol

#

you are making your own life harder doing it the way you are doing it

true vapor
#

No - I meant having them both in the same compose file

#

but yeah, I'll put try putting them both in the compose file again

wooden ibex
#

Yes, they should be in same compose file

#

When I get a moment, Iโ€™ll work on docker-compose

verbal linden
#

anyone know how to create a git ref for a github repo

finite fulcrum
#

Is there any way to make pycharm ignore "unreachable" code under an if __debug__ ?

tawdry needle
#

what do you mean by a "git ref"? @verbal linden

dim bison
#

Can someone explain what the difference in running the rdp file from a RSS web client and just rdping to the server ip is? The rdp file takes me into my vm pool but what about it does that?

wooden ibex
#
version: '3.1'

services:
  cheapestpintin_db:
    image: postgres
    restart: always
    environment:
      - POSTGRES_PASSWORD=docker
      - POSTGRES_USER=docker
      - POSTGRES_DB=db
#      - PGDATA=/var/lib/postgresql/data/pgdata
    container_name: cheapestpintin_db
    ports:
      - 5432:5432
    network: djangonetwork
#    volumes:
#    - ./volume/dbx:/var/lib/postgresql/data
  django:
    build:
        context: .
        dockerfile: django.dockerfile
    image: django
    restart: always
    networks: djangonetwork```
#

@true vapor

#

change db host to cheapestpintin_db or shorten the name

#

couldn't test since don't have repo but hopefully that helps

true vapor
#

Hm - am I supposed to do something else?

services.django.networks contains an invalid type, it should be an array, or an object
Unsupported config option for services.cheapestpintin_db: 'network'

is what I get after doing docker-compose up

wooden ibex
#

networks

#

not network

#

my bad while typing

#

line 15

true vapor
#
services.cheapestpintin_db.networks contains an invalid type, it should be an array, or an object
services.django.networks contains an invalid type, it should be an array, or an object

Do I need to change it to

    networks: 
      - djangonetwork```
wooden ibex
#

probably

#

I don't have my docker-compose with networks

true vapor
#

Got it working - you're fantastic, thank you

#

this saved me a lot of headache

wooden ibex
#

see if django can ping postgres

true vapor
#

There's some problems with my application apparently - need to fix those first. But this is much further than I made it before

wooden ibex
#

docker exec -it django ping <postgresimagename>

#

I prefer docker-compose build option, instead of trying to push a bunch of stuff into docker compose, just let it handle building and running

true vapor
#

that ping is working as expected yeah

#

thanks

wooden ibex
#

cool

true vapor
#

you honestly saved me so much time and fuckery that would have driven me insane. I have no idea how much time you spent on this, but you saved me probably hours - thanks a ton for that. One day I'll (probably) take the time to learn the basics of this stuff properly, but thanks for the help

wooden ibex
#

no problem

#

sorry it took a little bit, work was PITA when I logged in

true vapor
#

That's fine - I was mostly procrastinating with other stuff anyway, it was still super helpful

fleet tapir
#

Hello. I made a git post-receive hook as a bash script to get the latest commit information and format it with a python script. Then the message generated will be sent to discord through a webhook post request.
My issue with this is that when I test the python script out with some data instead of the sys.argsv[] that get passed through the bash script, the message is sent correctly, in that it sends the two messages generated.
But when I use the hook itself and push something, it sends only the first message without the changelog. I split the messages to send multiple messages that fit in the discord message limit.

#

Here is a screenshot of how the message is being sent. The first messages are sent correctly with the header and the changelog text (modified readme) which are sent directly from the python script . The ones after are triggered through the shell script of the hook and are missing the second message with the modified readme line.

#

This is the code that sends the message:

def Send(self,messages):
    count = 0
    for message in messages:
        count += 1
        content = self.template.replace("$content",message)
        requests.post(self.hook, data=content.encode('utf-8'), headers=self.type)
        if len(messages) > count : time.sleep(1.5)

I don't have much experience with the requests library or networking in general. When debugging, it seemed that the message seems to be correct and is being sent, but it never arrives to discord and it doesn't throw out any error either.

gaunt tundra
#

hi

#

can any one help me with my code i don't understand why it dos not work ...

#

any help ?

#

i will be tankfull

tawdry needle
#

@gaunt tundra go ahead and ask your question, dont "ask to ask"

#

we have a guide for asking good questions:

#

!ask

#

hm. where is the bot?

gaunt tundra
#

idk

#

!ask

tawdry needle
#

anyway... try to include your code (as text, not as a screenshot) and the full error output if you get an error. please also try to describe the output you wanted, and how it is different from what actually happened.

gaunt tundra
#

it's not a python error but the module that i used in the code dos not act like i want to ....

#

i will send the code just a sec

#

#!/usr/bin/env/ python

import pynput
from pynput.keyboard import Key, Listener
from pynput import keyboard

keys = []
#bad_keys = [Key.enter, Key.space, Key.alt_l, Key.esc]

def get_key_name(key):
if isinstance(key, keyboard.KeyCode):
return key.char
else:
return str(key)

def on_press(key):
keys.append(key)
write_file(keys)

def write_file(keys):
with open('log.txt', 'w') as f:
for key in keys:
k = str(key).replace("'", "")
f.write(k)
#f.write(' ')
def on_release(key):
if key == key.esc :
return false

with Listener(on_press = on_press ) as Listener:
Listener.join()
Listner.start()
controller = keyboard.controller()

sour burrow
#

I need help with something. I have a python program that schedules jobs using the sched module. The way i have the scheduler working right now is that it uses time.sleep() to wait until the time for the next job. Now I want to implement a way to get the status of the scheduler. So my question is how can I implement a way to ask for a status? The best would be if i could run "python program.py -status" and it would output something like "waiting until x time to start y task". First time on this discord let me know if i need to ask somewhere else. thanks

tawdry needle
#

@sour burrow it sounds like you need 2 separate programs running: a job scheduling daemon to act as a server, and the client that lets you check the status and control the scheduler

#

otherwise you can have your program react to SIGINFO

#

or whatever that one is

#

yeah SIGINFO

sour burrow
#

how would the client communicate with the server?

#

without SIGINFO...

tawdry needle
#

you'd need some kind of protocol that communicates over a socket. its probably way too complicated for a simple application

#

in general if you have 2 programs that need to talk to each other, you need to use some kind of OS-level communication mechanism

#

which is why im suggesting maybe just using 1 single program instead

sour burrow
#

what about a pipe?

tawdry needle
#

how would a pipe help

#

like a unix named pipe?

sour burrow
#

yeah i dont know

tawdry needle
#

so you are thinking something like

python -m run_scheduler

then being able to run

python -m scheduler_status

interactively?

#

this is meant to be a daemon that stays running for a long time?

sour burrow
#

if thats possible, that would be what im looking for

#

yes. it schedules tasks for the next 24 hours, then calls itself so it loops

tawdry needle
#

the scheduler?

#

what OS does this run on? what do you mean "calls itself"?

sour burrow
#

this is linux. rasperry pi to be more precise

#

i can show you my current code, but it will probably need to be change quite a bit if im gonna make it work interactively

tawdry needle
#

i cant necessarily offer a solution but maybe someone can

sour burrow
#

ok. I'll look more into the server and client stuff. thank you

sly sleet
#

@sour burrow what about dumping the scheduler status into a file and then reading that

thorn berry
#

@everyone anyone interested in joining the TurboCharge package manager dev team please dm me

gusty willow
#

Please stop cross posting this

#

And surely, don't try to ping every person in the channel.

thorn berry
#

Lol

lone linden
#

Am i understanding this correctly, that if we use the flag --use-feature=2020-resolver, and we use pip install -r reqs1.txt -r reqs2.txt will resolve dependencies differently then pip install -r reqs1.txt and pip install -r reqs2.txt seperately?

tawdry needle
#

its more that the 2020 resolver uses a different algorithm that might produce different results from 2 separate invocations

lone linden
#

i did a test run in our ci pipeline, where i installed reqs in 2 commands

#

now i repeated it in one, and it fails

#

caught me by suprise to say the least

#

how does it make sense that the outcome depends on the amount of commands though, should it?

#

is it really determenistic if it depends on the amount of commands

#

i mean, it probably is, if you use the same amount of commands, still, it doesn't make sense to me

tawdry needle
#

maybe its still not resolving with currently-installed packages in mind?

#

good question. im sure the algorithm details are somewhere on github

sand thistle
#

autobuilds as a first time docker user

#

are a bit confusing

#

im a bit confused about the build rules

#

and the build context

gusty willow
#

build context == the scope of the build. So if you use . that means that the build process will process everything in the current directory. But if use a different context, then the COPY . . calls or whatever have different meaning. Lets say I set the build context to ./build/data, that means, when i use COPY or ADD that the "root" (or .) level directory that i can call from, is actually the hosts ./build/data. Generally, you don't see this a lot, but it is useful in certain situations.

#

build rules generally refer to limiting builds on certain git tags or branches

finite fulcrum
#

Is there a way to install only the dev dependencies through pipenv?

sand thistle
#

having a git error ```error: There was a problem with the editor 'vi'.

vestal pier
#

are you trying to use vi as your default editor? You may need to change your editor

#

@sand thistle

sand thistle
#

I did

#

yeah

#

it helped

#

but now I get a create mode 100644

#

which from what I'm reading is some meta data

#

what's odd is t of his was there before

vestal pier
#

are you getting an error?

sand thistle
#

no its working

#

i's just weird

#

and makes me nervous

vestal pier
#

haha, git status

sand thistle
#

i mean everything is fine

heavy knot
#

is there a way to compile a python file into a .exe/application

#

idk if this is the right help chat

west sluice
#

@heavy knot pyinstaller is great for that

#

@sand thistle 100644 is the normal git mode

#

====================

#

So I've heard that pipenv and Pipfile are no longer the state of the art for python build and dependency management tools

#

I've heard of a pyproject.toml file which is supposedly compatible with multiple different build systems?

#

My question is, if I started a new python project right now, what build/dependency system should I use?

warm pollen
#

pipenv tbh

#

IMO pipenv isn't really great, but compared to other dependency managers, it is somewhat good

#

poetry could also be a valid choice, but I know some people who never managed to make it work

toxic bough
#

how do I make requirements.txt lol

#

like if I do pip freeze it gives me a list of all the packages

warm pollen
#

You redirect the output

toxic bough
#

ya i got that

warm pollen
#

pip freeze > requirements.txt

toxic bough
#

yeah ik that but it gives me all the packages i have

warm pollen
#

I mean, yeah, it will give all your packages in the current environment

toxic bough
#

the only packages I have installed for this particular project is pynput and docx

warm pollen
#

Did you installed them inside a venv?

toxic bough
#

can I manually make a requirements.txt then?

warm pollen
#

Yeah sure

#

It is actually recommended, so you let the packages set their dependencies themselves

toxic bough
#

like should will installing through requireme
ah yes I was wondering whether it will also download other things required by it

#

as I have seen requirements.txt of other people with random things

#

do I have to mention the version of each package?

warm pollen
#

Yeah, it is better if you do

toxic bough
#

ok cool

#

Thanks!

#

I'm not able to sign into github in vs code

#

I use manjaro Linux and my browser is Firefox

#

basically when I try to commit, it will tell me to sign in and I follow the process but it fails

#

I was able to reproduce the same thing after installing the Github Pull Request extension

#

basically it will take me to github. I sign in and press continue

dusty maple
#

poetry is about as neat as pipenv mainly because of pyproject.toml but then again it's missing quite a few things

#

I've yet to write a C/Rust extension for Python that's managed by poetry

toxic bough
#

so i follow the "Didn't work?" instructions

#

but when I paste the token and hit enter, I get sign in failed: Error: network error

#

I'm stumped

dusty maple
#

Currently this is my .travis.yml setup. Since Travis CI creates virtual environments by default, shouldn't I use poetry to install tox to the said environment, then invoke that instead?

install:
  - pip install poetry coverage tox-travis
script:
  - tox
after_success:
  - bash <(curl -s https://codecov.io/bash)
#

Though I guess this avoids duplication from poetry installing dependencies at the top level and at the level of tox

toxic bough
#

I'm stumped
not anymore. finally signed in. I did open link with system handler this time. However that gave me another error (Writing login information to the keychain failed with error 'The name org.freedesktop.secrets was not provided by any .service files'). After installing qtkeychain and gnome-keyring, it finally signed in

heavy knot
#

Any recommendations on python specific or just general VSCode extentions?

#

Trying it out for the first time after using Pycharm a lot

#

so far got gitlens and rainbow brackets

#

Also, any extension that emulates Atom's Emmet autocomplete for html?

sand thistle
#

how does docker fit into a CI/CD pipeline

#

right now I have it set so that any pushes to my git repo, auto generate a new docker image

#

my goal is to then take that docker image and auto upload/run on my server

#

im using digital ocean, and I'm looking through their tuts right now

lost night
#

Hi, how do you guys work with venv in vscode? The problem I have is when I setup my vscode environment do use my .venv it tries to install pylint in it which is obviously not what I want.

#

Or is it correct to install the pylint in the venv?

#

It seems to be pylint is not something which should be in the requirements.txt, since it is not needed to run the project.

wooden ibex
#

It needs pylint for error checking

#

You can remove it from requirements.txt

lost night
#

Yeah I know what it's for, but is it really the proper way to just remove it from the requierements? Installing pylint and mypy install additionall dependencies which bloat the requirements and are a hassle to remove

wooden ibex
#

I mean, what else are you going to do?

west sluice
#

keep them out of the requirements.txt @lost night

#

just install them seperately

#

it's ok to add additional packages to your venv

#

unless you regularly run pip freeze > requirements.txt which isn't the best idea

lost night
#

Got it

dusty maple
#

@lost night You could also add the development dependencies to a separate dev-requirements.txt file

hoary ridge
#

you could add a first line in the dev-requirements.txt: -r requirements.txt that would include all the dependencies from the base requirements.txt file into your dev-requirements.txt

reef mica
#

So I'm getting this error in VSCode and don't remember it messing it up last time I used it

C:\Users\sevsd\AppData\Local\Programs\Python\Python38-32\python.exe: can't open file 'Untitled-1': [Errno 2] No such file or directory```
reef mica
#

I've just swapped over to PyCharm for now

rough marlin
#

I started getting into vim, and have found I like it enough that I'm wondering if there's a somewhat friendly CLI debugger option for python

#

how do pudb and other things beyond basic pdb stack up?

#

but I'm wondering if there are any more obscure or recent tools that i've missed beyond those mentioned in that roundup.

#

So far, I've only used pycharm for debugging python

#

and I think GDB once in the distant past.

tawny temple
#

When should separate jobs be used in CI vs. just adding more steps? In my use cases, using separate jobs hasn't really felt justified, considering it requires spinning up a new agent and checking out code again.

#

Even if semantically it may make sense to have separate jobs

#

Like, a job to lint and a job to run tests.

dusty maple
#

One of my projects that on Travis with tox has the lint and test stages combined for each Python version

#

I didn't really mind running pre-commit twice instead of just once but running them on separate jobs seems a bit better as it decouples responsibilities a bit more

next sentinel
#

'm using Python 3.7.8, flask

ModuleNotFound: No module named 'SpeechRecognition' appears when I try to run my project.

But it works well when I try it with python -m speech_recognition:

still not working when I import speech_recognition

strong falcon
#

I'm not sure how smart VSCode's python autocompletion is supposed to be but does it complete

 from pathlib import Path
 for p in Path('.').rglob('*'):
   p.```
for you? It should be rather trivial to infer the type of p here, yet I get no completion at all
dusty maple
#

I'd assume the culprit would be the fact that rglob doesn't actually specify its type

warm pollen
#

Was going to check the source, but my phone can't load 1.5k lines of code lol

dusty maple
#

Not quite VSCode, but since I use MPLS the behaviour is relatively the same, in this case, the language server was smart enough to infer that xs is a List[Path]

carmine pulsar
#

Where do I ask Git related question?

warm pollen
#

Here is fine lemon_pleased

carmine pulsar
#

In a Pull Request I submitted, the collaborator commented something
and I see the Resolve Conversation button
So, I agree with him in one comment and I disagree with him in other.
So, the comment which I agree, should I press Resolve Conversation?
and what about the disagree?
Whats the proper way to address this?

warm pollen
#

When you disagree, you'd usually comment on their comment

#

And usually (although some projects doesn't do that) when you applied a suggestion, committed and pushed it, you'd mark the conversation as resolved

carmine pulsar
#

I kind of disagree but I don't really mind, if they keep the changes as it just a grammatical related issue.

#

In one single commit, I made two changes. Of which one they disagree and one they agree.

So, in which I disagree but really dont mind, what should I comment exactly. I dont want to sound rude or bad.

The one which I agree, I will mark it as resolve conversation.

warm pollen
#

You should just explain why you are disagreeing with them

#

Disagreeing isn't a bad thing, it is a good thing actually

#

To maintain a good code base, you have to get thoughts from many people

carmine pulsar
#

The thing is its a grammatical issue. So they could be right too.

warm pollen
#

Although remember that they are the maintainers, so they have the final word

#

Well, that's up to you

#

Just explain your point of vue, why you think your wording is better

carmine pulsar
#

In another scenario, consider, I found I was wrong with the grammatical issue. Then what should I comment to sound proper and not too apologetic

#

I want to maintain etiquette

warm pollen
#

Tbh, if that's just a grammar issue, you don't need to apologize or whatever

#

It just happen, code reviews are here for that

carmine pulsar
#

And to make that disagreed change back to original, they will first have to Request Changes right?

I will ask them to request changes then?

warm pollen
#

They just commented, not requested changes?

#

Well, either way it is just a way of noting if you are favorable to the PR yet or not, it doesn't really mean anything

carmine pulsar
#

Yeah they just commented

fossil parrot
#

Anyone can please let me know free or cracked software version of google map data crawler to extract data?

hushed plume
#

does anyone know how i can cd into a user folder in a batch file without stating the specific user? e.g %user%

warm pollen
#

You could use the %UserProfile% variable

wraith ginkgo
#

~

warm pollen
#

Or ~ yeah

hushed plume
#

thanks

sick smelt
#

Hello, can I ask totally newbie questions here or is there a better place to do so ? (I would like to know if there's a search or filter option on https://pythondiscord.com/pages/resources/ or a way to show them all)

lucid spade
forest bay
#

@sick smelt #community-meta would be a good place to bring up a discussion on that

sick smelt
#

@forest bay Thank you, I've realized that recentely ๐Ÿ˜‰

carmine pulsar
#

However, because you made a PR from YOUR_ORGANIZATION_REPO instead of YOUR_MAIN_REPO, I'm unable to force push to your PR branch (even if you checked the "allow edits" box). Would you mind adding me to https://github.com/YOUR_ORGANIZATION_REPO/their_repo_name so i'm able to do that, and land this PR?

#

I sent a PR.
This was their reply.
So does this mean, I should add them as a collaborator?
to this repo https://github.com/YOUR_ORGANIZATION_REPO/their_repo_name?
Like going to Manage access -> and inviting them with write access?

tawny temple
#

Sounds like it

true vapor
#

has anyone used GCP+django_storages? I'm trying to set my app up on GCP and there's practically no documentation for it, and I'm getting a weird error

#
you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.compute_engine.credentials.Credentials'> just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.

This <link rel="stylesheet" href="{% static "public/css/materialize.css" %}"> is the line that's causing the issue, it works fine locally with a GCP credentials json thing

vapid lance
#

hey can anyone help me with pyinstaller?

quaint saffron
quaint saffron
#

How do I make it so that when I start up the docker image via docker-compose up it will update the files on the image? Right now the new files aren't being copied over, it just sticks with the files originally sent when first composing.
Dockerfile:docker FROM python:3.8-alpine WORKDIR . ENV FLASK_APP main.py ENV FLASK_RUN_HOST 0.0.0.0 RUN apk add --no-cache gcc musl-dev linux-headers COPY requirements.txt requirements.txt RUN pip install -r requirements.txt EXPOSE 5000 COPY . . CMD ["flask", "run"]docker-compose.yml:yml version: '3' services: web: build: . ports: - "5000:5000" redis: image: "redis:alpine"

warm pollen
#

You can either bind mount the source code (which mean the compose won't be portable anymore), or rebuild the image every time you launch compose using the --build flag

#

Thanks to caching, rebuilding will be almost instantaneous, but you can still optimize your image a bit

#

Moving EXPOSE and CMD just after your two ENV would make it really instantaneous

quaint saffron
#

I wanted to make it eventually so that things will automatically be uploaded to the image as soon as they're edited (and saved), without me having to compose/build every time

#

I believe you have to use a volume or something for that but not gotten that far yet

warm pollen
#

Yeah, you could use a bind mount, but again it means that you will have to have a local copy of the code in order for the stack to work, which you'd probably not have in production

quaint saffron
#

So would I not want to have a bind mount then?

warm pollen
#

Some prefers to rebuild, some prefers to have two compose, that's up to you

#

Flask doesn't auto-reload code like django does, right ?

quaint saffron
#

Tbh I won't actually be using flask, just the tutorial I was following has that as example

#

I'm using redis for a discord bot in the long-term

warm pollen
#

Okay

#

Because if your app supports auto reload you'd want to use a bind mount, but if it doesn't that's really up to you

quaint saffron
#

How would I know if it supports auto-reload or not?

warm pollen
#

I meant like django, if you modify the source code of your application it will reload the changes

#

But I'm 99% sure that d.py doesn't do that, does it?

quaint saffron
#

Dpy doesnโ€™t, correct

#

You have to either restart bot or restart the cog

warm pollen
#

If you bind mount a folder, any change that is done outside of the container will be done inside the container too, and vice versa

#

So it would allow you to hot reload the cog actually

quaint saffron
#

So like when a file is uploaded itโ€™ll execute bot.reload_extension('cog')?

warm pollen
#

You could do that by watching file changes, yeah

#

For that you'd need a bind mount

quaint saffron
#

Right okay

#

Iโ€™ll do that then

#

Do you have a link to like a tutorial of how to do that? (Setup bind mount and watch for file changes)

warm pollen
#

The docs for this are kinda bad though

#

What you'd have to do basically is firstly to put your app code inside a subdirectory

#

You could for example use WORKDIR /app before your first COPY

#

And then add a bind mount inside your compose using something like

volumes:
   - .:/app```
quaint saffron
#

Alright, thanks ๐Ÿ‘

#

Iโ€™ll check those out

#

Just to verify, I'd have the pywatchman on the docker image right? Cause files edited on my side would automatically be uploaded by the bind mound @warm pollen

warm pollen
#

Yup

#

By the way, Docker is creating an actual bind mount, it works the same way as you'd mount --bind any other folder

quaint saffron
#

I've kinda come back round to my original problem now

#

I've changed the config file to add the mount, but I can't actually 'upload' the new structure of code

#

Would the easiest way to just be delete the current container and recreate it?

warm pollen
#

docker-compose up --build --no-cache

#

But I don't understand your problem though

#

Using just --build it should go through the build steps again

quaint saffron
#

Right yea, I forgot the --build option

drowsy lava
#

through sublime's build feature I am able to run the discord bot however there is no output

#

This is my build system file

#

Is it because I am using a custom build system?

#

the problem is fixed but I can't stop my bot from sublime

#

there are two instances of my bot open

warm pollen
#

@quaint saffron can you-- welp, did you solved it?

quaint saffron
#

yea I had the FLASK_APP env as /code/main.py instead of code/main.py

#

Although I got another issue

#

The mount doesn't seem to be working

#

I make a change, refresh the webpage, and it doesn't update

#
version: '3'
services:
  web:
    build: .
    ports:
      - "5000:5000"
    volumes:
      - .:/code
  redis:
    image: "redis:alpine"```this is my yml file, so I have setup the mount
#

And I built the image with docker-compose --build

warm pollen
#

Hmm

#

If you kill and restart the container without building, will the web page change?

quaint saffron
#

And it is uploaded the updated docker-compose.yml file to the container C:\Users\tizzy>docker exec -it redis_web_1 cat docker-compose.yml version: '3' services: web: build: . ports: - "5000:5000" volumes: - .:/code redis: image: "redis:alpine"

#

lemme try that rq

#

Restarting the container does update the code (using ctrl+c to close script and then docker-compose up)

#

But the mount part doesn't appear to be working

warm pollen
#

You restarted it without the --build option, right?

quaint saffron
#

correct

#

Just docker-compose up

#

So it seems to be uploading the new file but not restarting?

warm pollen
#

Then the bind mount is working, my best bet is that flask is doing some kind of caching

quaint saffron
#

I can't find anything about caching on the flask docs

warm pollen
#

Maybe it was your browser doing some caching?

quaint saffron
#

Closing browser and reopening makes no difference

#

Nor does a hard-refresh (ctrl+f5)

#

And nor does using a completely different browser

#

So don't think it's the browser-end

#

I can confirm the file is definitely being sent to docker though, cause I can cat the file and it shows the new content

#

Just flask doesn't seem to be restarting, like it should (I know it's not restarting cause it'd show in logs if it was)

#

@warm pollen any ideas?

#

All I can really think of is using the pywatchman you linked earlier to restart the flask app

warm pollen
#

Sorry I got busy

#

Yeah, you could try that

rough marlin
#

Do computers count as development tools for the purposes of this channel?

quaint saffron
#

Right okay so now I need help with installing pywatchman. I'm not sure which container I need to install it onto @warm pollen

#
C:\Users\tizzy\PycharmProjects\redis>docker-compose ps
    Name                   Command               State           Ports
-------------------------------------------------------------------------------
redis_redis_1   docker-entrypoint.sh redis ...   Up      6379/tcp
redis_web_1     flask run                        Up      0.0.0.0:5000->5000/tcp```I've got two and not sure which to do it to.
warm pollen
#

redis_web_1, the python container

quaint saffron
#

Right, okay

quaint saffron
#

How do install homebrew onto the container? @warm pollen

warm pollen
#

Uunnnhh

#

Why do you want homebrew? lemon_thinking

quaint saffron
#

Apparently I need to install the watchman thing onto the linux distro

rough marlin
#

for docker you use different package managers

quaint saffron
#

I'm getting confused with how to use pywatchman because there's literally no docs

rough marlin
#

depending on what the base distro you're using for a container

warm pollen
#

There's some docs on the project homepage

quaint saffron
#

It just links to watchman docs which links to another watchman docs thing

warm pollen
rough marlin
#

a debian/ubuntu based image will use apt, fedora-based ones will probably use rpm

#

homebrew is a mac version of the same idea as the linux package managers i mentioned above

warm pollen
quaint saffron
#

gtk, thanks @rough marlin ๐Ÿ‘

#

If you go to the docs section on that page is says to install with brew install watchman @warm pollen

rough marlin
#

they mention building from source under the linux/mac section

warm pollen
#

You'd install the python binding through pypi here

#

Might as well look at the django source

#

Hold on

rough marlin
#

not sure if you need watchman inside a docker container, but which package manager you're using for each task will depend on where it's running and whether it's installing python packages or system ones

quaint saffron
#

I don't know how to install those onto docker though, that's onto the machine I'm on @rough marlin

rough marlin
#

so, before we go any further

#

i think you should know a bit about how docker works

#

You're used to using a mac, right?

quaint saffron
#

Windows normally

#

Never used mac/linux

rough marlin
#

are you on mac right now?

quaint saffron
#

Windows

rough marlin
#

ah, ok

#

Oh, I see.

#

ok, the documentation they have isn't very good.

#

I found it confusing too.

quaint saffron
#

Never used linux stuff before which is why I'm struggling so much lol

rough marlin
#

That's ok.

warm pollen
rough marlin
#

Each OS generally has something called a kernel

#

that's the heart of the OS, everything else plugs into it.

#

Windows has one too. The current windows is a descendant of the NT kernel from the 90s, OS X / macOS is sort of a mashup of BSDs and other stuff