#tools-and-devops
1 messages ยท Page 52 of 1
lol
i went through the logs but it's all gibberish to me
the ufw logs
I see a bunch of UFW blocks
what happens if you just disable ufw temporarily
obviously, don't leave it that way, but that would eliminate your firewall as the actual problem or not
ok, so that eliminates UFW as the problem
and does it quit if you turn UFW back on?
ok, well good ... we have eliminated a potential cause of the failure ๐
how many workers are you running?
because i've seen it to where not all of the workers were able to access the entrypoint before (and that is some weird stuff to troubleshoot, let me tell you)
but arent they separate entry points
hmm so rebinding gunicorn helped
well
fuck
lol what now
just fyi, be mindful of the language
k
yeah idk
my systemd file is messed up or something
instead of using a unix socket though
I'm using tcp
and doing localhost:8080
i mean, that should work
connection refused
systemctl status shows that there is a pathing issue
Failed to execute command: No such file or directory
well ... if you can't even do the localhost curl, then gunicorn isn't running
are you trying to run gunicorn with systemd?
i may have missed that bit if that is the case
the systemd file?
yeah
my guess is you probably just need to provide the full path to the gunicorn executable
[Unit]
Description=Gunicorn instance to serve expDashboard.py
After=network.target
[Service]
User=expadmin
Group=www-data
WorkingDirectory=/home/expadmin/ExpFlask/managment_dashboard
Environment="PATH=/home/expadmin/ExpFlask/managment_dashboard/expDashboardVenv/bin"
ExecStart=/home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn --workers 3 --bind localhost:8080 -m 007 wsgi:app
[Install]
WantedBy=multi-user.target
yeah but
doesnt say that
it's telling me to put the location of the .py file, and the path variable
and then using gunicorn
which would be stored in the bin file
does expadmin/www-data actually have permissions to that environment?
it may not
ls -lash /home/expadmin/ExpFlask/management_dashboard
my guess is, you are going to be seeing expadmin:expadmin on ownership
so i accidently cd into the directory
but
everything has root except
the venv folder
so it's all root:root?
which is where gunicorn is actually stored
expadmin:expadmin for that one
as well as
expadmin for ..
ok, so there are no folders that the "www-data" group actually has permissions then
so that is your problem
generally, i don't like to run sites out of /home...
i tend to use /var/www or something of that nature
i see
this is my first deployment
so then I need to switch permissions for which folders?
chmod
in your case, you first need to make sure that your user is a part of that group sudo usermod -aG www-data expadmin && newgrp www-data
then, make sure that the folders that are serving all of that have the correct permissions sudo chown -R expadmin:www-data /home/expadmin/ExpFlask
erm I'm missing an operand
and for good measure, i would set the permissions like so sudo chmod -R 0770 /home/expadmin/ExpFlask
if the group www-data doesn't exist, you may get an error ... in which case sudo groupadd www-data
but that is easy to find if you just grep www-data /etc/group
www-data:x:33:expadmin
ok, so the group exists, that is good, but yeah, do the chown/chmod stuff and then try to run your .service file
then, make sure that the folders that are serving all of that have the correct permissions
sudo chown -R expadmin:www-data /home/expadmin/ExpFlask
@gusty willow am i supposed to specify the folders here, afterExpFlask
the -R means recursive, so anything under that will get those permissions
ah sry
discord messed up the syntax a bit
yeah so
i did all that, and double check permissions
different error
now it just wont start
what error
Main process exited, code=exited, status=203/EXEC
before it couldnt locate the file
right, so it ran, but couldn't find anything, now it can but it's failing for whatever reason ... we are getting closer ๐
can you add --log-level debug to your .service file
and try again
maybe we can get more info
also, do you actually have gunicorn installed the venv? not just the global?
it's in my requirements.txt
and gunicorn is located in the venv folder as well
under bin
just making sure ๐
can you add --log-level debug to your .service file
@gusty willow this would be under the service portion of the systemd file
but yeah, in your ExecStart line, add --log-level debug to the end or wherever you want to add that particular argument
again, docker solves a lot of these particular issues lolol
hmm
I added this after the end
--log-level debug
--workers 3 --bind localhost:8080 -m 007 wsgi:app --log-level debug
now its the same error as prior
expDashboard.service: Failed to execute command: No such file or directory
we just fixed this error
maybe it needs to go before the -m
I may have entered it incorrectly
i'm checking the syntax real quick ๐
yeah, it's just --log-level LEVEL
but yes, it does need to be before your wsgi:app declaration
my bad
do you have your code somewhere in git that I can see it all in totality
I hate everything
do you mean my systemd file, do I have that on github
no
ExecStart=/home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn --log-level LEVEL --workers 3 --bind localhost:8080 -m 007 wsgi:app
you need to do --log-level debug ๐
and also, looks like you have too many spaces after the level
same error
and what happens if you do newgrp www-data && /home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn -h just from your terminal
nothing happens
what about just running /home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn -h
my suggestion here is to probably pip install --user gunicorn and then adjusting your .service file to call /usr/local/bin/gunicorn rather than from inside the venv though
so
umm
this
/home/expadmin/ExpFlask/management_dashboard/expDashboardVenv/bin/gunicorn -h
yields
no such file
ooo, ok, so there are problems lolol
so, truely, if you want my honest opinion, run a container, it'll solve all of this
i was advised heroku as well
heroku would work, but it's a bandaid for one off things
to learn how to really do deployments, i wouldn't learn that
do you have docker installed?
i dont
i can walk you through this pretty easily after you get it installed
youre suggesting i use a docker container in my droplet?
yeah, i do it all the time ๐
there are docker base images too
for droplets
that might be easier
idk if DO has a direct "run this container" option, but if they do, that would be best lol
im installing it
and does your code exist on that droplet? i assume it does
im very frustrated
When creating a python script which needs an api key, what's the best way to provide it to the script?
Python config file, JSON config file, enviroment variable or command line argument?
I've never used a mac, but im curious if the default terminal is bash or has bash-like commands/keystrokes.
Also, I assume vim is pre-installed?
default terminal is bash @covert kindle
i use nano so I can't say, but I believe vim is preinstalled
Awesome, thanks!
@lost night I would argue that none of them are preferred over a dedicated secret store
@lost night but of those four, envvars or arguments make sure you aren't going to git commit secrets
@covert kindle I can't remember if the preinstall is vi or vim, but it's one of those lol
it's not vi
@sand thistle
Dockers are an awesome tool, I use them daily for development and for random testing. They come in super handy since I use windows but most of my tools are unix. I even used them to replace an old FreeNAS build.
The docker classroom is great for getting started. Definitely a tool that is worth learning, especially with its many uses.
Play with Docker Classroom https://training.play-with-docker.com/
Learn docker through online trainings in training.play-with-docker.com
I used them for deploying a Django/PostgreSQL/Redis/NGINX/gunicorn/Daphne project. Worked great.
what's daphne?
thanks @gusty willow
Asgi webserver iirc
if im wanting to run a app that uses python and js do i need to build my docker file from base image or can i use the python image and install node in there?
Both will work
Using a base image is just more convenient
As in, using a node or python image as a base is more convenient than creating one yourself
im just not comfortable with base images cuz idk how to specify python version. seems like the only thing i can ensure is that the python is 3 not 2 lol
What do you mean by base image
like from os
cuz when u run install python3 u kinda cant specify the .x at least from what i know
It depends on the OS I suppose. Usually you don't use a rolling release style OS for images, so you're "stuck" with what the OS package index has, unless you want to build and install python from source or use a tool like pyenv. That's why using a Python image as a base is better than a bare OS image.
i was gonna start with bare os and install python and node
but it seems idk how to specify the python version
like u can only do install python3 not sth like install python3.8 or sth
Why not pick one as the base and install the other
yeah i think that prob much easier
but just curious how do u specify python version tho
See the message I sent 2 messages ago
The same thing applies to Node, so you could run into the problem anyway if you need a newer or older version than the OS has.
Unless you find an image with both Python and Node that has the versions you need.
Granted, I'm not really able to imagine how a single app could rely on both Python and Node. Are you sure you're not trying to stuff 2 separate apps into one container?
log into what?
And run what
You can run a shell in the container, which will let you execute arbitrary commands, if that answers your question.
@tawny temple I was working with him, he meant logging in with docker login and then running the container
I see. I'm not sure why you'd need a login before a run, but I suppose you could do it with basic shell syntax docker login && docker run ...
Oh right, run will pull if it doesn't exist
There are so many CI/CD products and I can't decide which one to use
Jenkins seems to be both the most hated and loved...?
I don't get the hate for Jenkins. It does what I need. The interface looks dated, but the shit works just fine.
how do i use a virtualbox
i need to use linux for python, as i will compete in hack the box (an online place to hack people's vm)
superuser, you wouldn't use the same container, you would have python container and JS container
I'm looking for a website where I can upload a simple install script, so I can curl | bash it, but I'd like the url to be pretty small, and github raw urls aren't short at all, do you have any site to suggest?
I have the github student pack,I'm trying to sign up to aws but I don't have a credit card, what do I do?
I'm 'ot sure about how long pastebin stores pastes
I guess an url shortener could work
It is intended to be public, so I was wondering if they were a service dedicated to that
github?
They have really long urls
im surprised there isnt some "file hosting for hackers" site with short urls
is a thing, but it's on tor lol
hmm
im thinking of just a dumb http/ftp server where you can upload up to say 10 mb of files
with some up/down bandwidth cap
and short cute URLs
nothin like that exists currently?
people would use it for bad stuff most likely
Akarys, use url short to point to github raw?
Yeah, I guess I'll do that
@wooden ibex you can use github for bad stuff too though
i heard thats why firefox send was pulled, but i dont see how this isnt an issue for literally any other site that hosts user provided data
anyone here with an experience of scraping pdfs? i want to parse pdf tables with very odd formatting, there are 2 types of column in one table (!) each of different width, multirow cells and also multicolumn cells, on top of that tables span across multiple pages
any ideas how to approach this task?
@heavy knot ive used https://tabula.technology/ before, but idk how well it will work in your case
Tabula is a free tool for extracting data from PDF files into CSV and Excel files.
@tawdry needle thanks, i'll give it a try, I tried tabula-py already and didn't have much luck with it
well, it works better than python version, still doesn't store multicells correctly like i want a value to be stored in each subcell
as per its github page, it is not actively maintained
luckily it's fork is https://github.com/pdfminer/pdfminer.six
now i have one more option to try ๐
These PDF libs may already work this way, but consider exporting the pages to images and then using a library that can extract tables from images rather than PDFs. May be worth a shot.
@tawny temple thanks for suggestion, that's an interesting approach, i didn't thought about it
my main problem with pdf libs is that if there are multicells as well as empty cells on a single line it is hard to distinguish between NULL values from them
maybe those OCR libs will be able to deal with it more effectively
if python IDLE and Atom had a baby what would it be?
@fathom silo vs code?
since vs code has a python language server and repl functionality built in
sublime text supports LSP and has sublime REPL but its honestly not as good as vs code
and there are always more "full featured" IDEs like pycharm
can you run code directly off of it
yeah, i dont use vs code myself but there's some setting or keyboard shortcut to run a whole file or a line from a file
that is literally all I wanted thanks
ive been thinking of switching to vsc now that their python support is better and sublime text plugin development is kind of in a rough state
rip sublime text community ๐ฆ
it's 70 bucks now right?
its always been $70
oh
the problem now is that the plugin dev community is nowhere near as active as it used to be
so its ripoffs (atom and vs code) are starting to surpass it
no idea where they went, i think people just switched away from sublime
the devs went silent for basically years before 3.0 was out of beta
sublime is a rock solid stable editor still and the devs earned by $70 many times over
but its sad to see the plugin ecosystem starting to wither
lots of years-old unmaintained plugins now
i think they should just open source sublime or something, idk
you know if you can run code directly from atom without any plugin?
im not sure, atom might have some kind of repl functionality built in but i wouldnt know
just googled there's an extension with 1.8m installs that does exactly that
"script"
is an extension not the same thing as a plugin?
vs code is just adding features at a relentless pace and they place special emphasis on python
yeah vs code is very heavily funded by msoft right now
putting a lot of work into typescript too i think
and the language server protocol itself
why do they do it tho
I would understand if it wasn't available to linux but how do they profit off of it?
i'm trying to host my discord bot on azure, I SSHed and typed python <filename> it runs fine but when I close my terminal it kills the process, why?
@fathom silo i think its basically a big PR move
embrace extend extinguish except they arent necessarily going to extinguish
vs code is a gateway into a whole new "dev friendly" microsoft ecosystem
oh I see
im not sure what else ms gets out of vs code specifically
but i know that supporting e.g. typescript helps ensure that typescript is good enough and usable for their purposes
@oak path if you quit your ssh session your process receives a hangup signal from the server resulting in killing your process
You could try nohup
what is that?
something like nohup <process> &
okay
i use tmux
nohup - run a command immune to hangups, with output to a non-tty
@tawdry needle with detaching the session after running the process? this will also work
if I want to run a jar file on the side what will be the port?
salt rock lamp, Azure
VSCode is gateway drug to Azure
also, Microsoft needed editor for their Python/Java programmers
and Rust, can't forget that
if that isn't an indictment of how horrible visual studio is, then i dont know what is
Nothing
Visual Studio has always been designed for .Net programming
it has plugins for other stuff but Python/Rust/Javascript have always leaned towards more lightweight editors and Microsoft realized providing said lightweight editors was key to winning them so they did it
We do .Net at work, Once you get over learning curve of Visual Studio, it's ultra powerful (like PyCharm)
for C/C++ it works well
for Python/Javascript/Typescript, I believe VSCode is most popular on Windows
well for python pycharm is still quite popular. i dont know anyone who uses visual studio for python
I mean, there is a plugin for it.....
is this the channel for conda related question too?
sure, conda is on topic here
In conda how can I remove only packages installed from a certain channel?
the other one is how can I clone the base environment as it like when conda is newly installed which I mean not to include those new added packages (which might also mean the rev 0 in conda list -r)
@oak path no idea what you are talking about
Np
@oak path no, I mean what are you talking about lol. Jar files are java executables that would be configured in their own way, so there is no way we could tell you what port that app runs on. And how does that relate to python anyway? I'm just generally confused.
@ebon spire the last column of conda list output is the "Channel" column. you can use --show-channel-urls to get more detail in that column, and of course you can use --json to pass it to another program like JQ
read -r -d '' jq_filter_prog <<'JQ'
del(
.[]
| select( .channel != "pypi" ))
| map(.name)
| "conda uninstall " + join(" ")
JQ
read -r -d '' uninstall_command < <(conda list --json | jq -r "$jq_filter_prog ")
echo "$uninstall_command"
@ebon spire if you want to get extra-slick using jq ๐
as for the 2nd question, im not sure. are you trying to "reset" the base env?
hello, assuming i am running a script on a hoster, and this script is downloading files to its root directory that i need to keep - the hoster, however, resets daily. what is the simplest and best free solution to storing these files?
i know about aws s3 but that doesnt seem very simple to me
Unfortunately, that might be the solution you have to use if you want to work with that platform... S3 gives you 5GB for twelve months for free, if that makes it any better. If your files are small enough/formatted consistently, you can store them in a database
What platform are you using?
s3 is pretty straightforward for a single user. just make sure it's set to private, not public
you can also use backblaze b2, i have good experiences with that company
or google cloud store, or azure blob store, etc.
If you're on Heroku, here's a pretty good S3 tutorial with a Python specifically: https://devcenter.heroku.com/articles/s3-upload-python
You can also have a look at the AWS docs: https://docs.aws.amazon.com/AmazonS3/latest/gsg/GetStartedWithS3.html
Get started storing and retrieving any amount of data at any time, from anywhere on the web with Amazon S3.
Does anyone have a decent production ready Django Docker setup I can look at?
preferably a relatively simple one with no bells or whistles
i dont have one, but you'd typically be looking at just python3 + django + gunicorn + your app, right?
https://www.caktusgroup.com/blog/2017/03/14/production-ready-dockerfile-your-python-django-app/ Here's what I've been working with so far
There's a lot of random small but important considerations
# Your settings should already be inside ./app/, or use the Docker --env--file option
# Include Django and Gunicorn in requirements.txt
FROM python:3
RUN mkdir /app
RUN python -m venv /venv
COPY ./ /app/
WORKDIR /app
RUN /venv/bin/pip install -r requirements.txt
RUN /venv/bin/pip install .
EXPOSE 8000
CMD ["gunicorn", "--bind", ":8000", "--workers", "3", "myapp.wsgi:application"]
this seems like the absolue bare minimum django+gunicorn setup, assuming your python app itself is sane
if you want to be really slick, you can (should) use pip-compile to generate the requirements.txt file from requirements.in
Yea that link is awful, when you are install deps shit is going sideways
yeah, installing deps by compiling from source... yikes
oh its not from source but still
@wooden ibex aren't there times when you just need to use the package manager in docker?
maybe this is just them wanting to keep the container smaller by using FROM python:3.7-slim and then installing only what they need
Which is dumb
im more alarmed by all the hard-coded env stuff in the dockerfile
Thatโs standard developer dumb which SRE will come in and beat with a stick
how do you feel about changing USER inside the dockerfile? i just added that to my snippet but i dont typically see it
so i wonder if there's some reason people dont normally use it
RUN groupadd -r ${APP_USER} && useradd --no-log-init -r -g ${APP_USER} ${APP_USER} seems weird and too "manual" for docker
this seems like a much better tutorial @true vapor https://www.digitalocean.com/community/tutorials/how-to-build-a-django-and-gunicorn-application-with-docker
Yea that link is awful, when you are install deps shit is going sideways
That is very sad news
That looks like a much better guide
salt rock lamp, it's considering best practice because running as root
thats what i figured. so you need to manually create the user inside the dockerfile if your image doesn't do it for you?
@tawdry needle NO
oh lol
This is the suggested dockerfile from the DO article - I don't honestly understand it, but that RUN set seems funky
ADD django-polls/requirements.txt /app/requirements.txt
RUN set -ex \
&& apk add --no-cache --virtual .build-deps postgresql-dev build-base \
&& python -m venv /env \
&& /env/bin/pip install --upgrade pip \
&& /env/bin/pip install --no-cache-dir -r /app/requirements.txt \
&& runDeps="$(scanelf --needed --nobanner --recursive /env \
| awk '{ gsub(/,/, "\nso:", $2); print "so:" $2 }' \
| sort -u \
| xargs -r apk info --installed \
| sort -u)" \
&& apk add --virtual rundeps $runDeps \
&& apk del .build-deps
ADD django-polls /app
WORKDIR /app
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
EXPOSE 8000```
do we hate alpine too? @wooden ibex
set -ex is just a bash thing to stop it from continuing if one line errors out
FROM python:3.7.4-alpine3.10
dont install postgres into your container tho...
that much i know not to do
and this is bad/weird, just use /env/bin/python and /env/bin/pip explicitly
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
what salt rock lamp posted looks good
the DO article?
no
just the tiny dockerfile?
# Your settings should already be inside ./app/, or use the Docker --env--file option
# Include Django and Gunicorn in requirements.txt
FROM python:3
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
EXPOSE 8000
CMD ["gunicorn", "--bind", ":8000", "--workers", "3", "myapp.wsgi:application"]```
from Salt rock lamp
yeah, sure. It misses out some stuff though, mainly migrations and staticfiles which I have no idea how to handle in production. Although that isn't in Docker's domain I don't think
@wooden ibex some python people feel strongly about using it anyway
at least, #python on freenode does
I mean, it's not end of the world but whatever
they probably think Alpine is good idea too
isn't alpine just sort of objectively bad?
When youโre choosing a base image for your Docker image, Alpine Linux is often recommended. Using Alpine, youโre told, will make your images smaller and speed up your builds. And if youโre using Go thatโs reasonable advice. But if youโre using Python, Alpine Linux will quite o...
this is good to know
Why? Most Linux distributions use the GNU version (glibc) of the standard C library that is required by pretty much every C program, including Python. But Alpine Linux uses musl, those binary wheels are compiled against glibc, and therefore Alpine disabled Linux wheel support.
ew
suckless, amirite?
nah bro it sucks less!
i too recompile my entire window manager from scratch when i change settings, because config files are bloat
I just use Win10 and problem solved 
BTW Charlie, I edited salt rock lamp docker file
see if that works
It's not gonna work right now - I don't have a database set up and running locally
but that's what I currently have in my Dockerfile
this is what docker compose is good for right?
1 dockerfile for database, 1 dockerfile for django app
You'd usually pull the database container from dockerhub though
nginx supports binary streams now too right? so you can put them both behind nginx and have a nice time
nah, I would just have django app make a call to database over internal docker network
i mean if you also need to expose the database outside
Right now I have an attrocioius setup on a OVH VPS using some monstrosity dockerfile and some monstrosity DB running in a dockerfile. I'm trying to get it set up in a way that I can just plonk it on GCP and use a cloud SQL db
@warm pollen im actually curious about how that works, do you have any setup examples?
salt rock lamp, you can configure docker to have internal only networks
The repository for the 2020 summer code jam. Find more information at https://pythondiscord.com/pages/code-jams/code-jam-7/ - python-discord/summer-code-jam-2020
- ./config/nginx:/etc/nginx/conf.d
- ./static:/static``` 
BTW, you can have docker compose use docker files for build
drblistener:
build:
context: .
dockerfile: discord_listener.dockerfile```
FROM python:3.6-slim-buster
# If you prefer miniconda:
#FROM continuumio/miniconda3
WORKDIR /app
COPY Modules/ Modules/
COPY parameters/parameters.json parameters/
COPY requirements.txt .
COPY discord_listener.py .
# Using pip:
RUN python3 -m pip install -r requirements.txt
CMD ["python3", "./discord_listener.py"]```
Riiight
No I get while you are using volumes, I just not a fan of them for stuff that should be checked into git, shoved into container
volumes should only hold data you can't rebuild (like database files)
Yup I agree with that actually
I wish Jams would let SREs participate
code jams?
Site reliability engineers
People like j4ng5y and I and others who would be cranky about docker files
๐
You could participate. Of course it's jam dependant, SRE stuff isn't at all relevant for Kivy desktop apps, but having someone handling all the SRE stuff for Django would probably have been useful in the last jam
Charlie we would occupy a slot going to developer
There's 5 people in a team, usually up to 3 are inactive
so - most teams wouldn't complain too much
you could always mention it in meta - suggesting that some amount of emphasis be put on stuff outside of code
Yeah, I personally wouldn't have minded
Code, though obviously important, is usually only like 40% of the game lol
making it run is arguably more important lol
chicken vs egg though
God I hate this devops side of things. I think this Dockerfile works, but now I need to write a damn docker-compose file for the database, and make sure my app is getting the database creds correctly
and
I just don't care
I know the code works
I wrote it
but I have to care :(
said every developer in the era of devops lol
shift left they said, it'll be great they said
but really, i spent a good chuck of my time at my last gig just making dev lives easier in that regard (i ultimately ended up suggesting and deploying skaffold bundled with k3s with some template repos that had basic manifests for common tasks)
there also isn't a need for every team to have SRE, most of time once we setup docker and github actions, it's done
yeah, we had 300+ engineers and a SRE team of 5 and we managed just fine lol
gitlab here, but same thread
@modern stump yea heroku, ive been thinking that i can use dropbox but i dont know what the limits there are, amazon seems to have a pretty complicated setup and apparently needs verification via credit card?
@mystic flint not sure what you are referring to, but using dropbox as "app storage" doesn't sound like a good idea lol
its just for a bunch of small files (mp3s) that are like 2-5 mb in total
if you don't care about speed, then yeah, probably ok. But don't expect to use that on a resume ๐
how does one set the python path? i'm setting up an espressif project in visual studio and it complains about how it can't recognize pip/python in my environment
this didn't work```
PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin/${PATH}"
export PATH ```
@devout temple first of all, you set the path incorrectly ๐
assuming that is the correct filepath to your python binary: should be export PATH=/Library/Frameworks/Python.framework/Versions/3.8/bin:${PATH}, you missed the :
most of the time though, i just suggest people use brew on a mac
right...but does it set PYTHONPATH?
have you used visual studio? I changed the path for python but it still seems to be using an older path that's inaccessible
okay so I put Library/Frameworks/Python.framework/Versions/3.8/bin/python3 as the python absolute path but it still complains `Python or pip have not been found in your environment.
`
ayo how do i make a script run periodically
Cron if you're on Linux
hello all, i don't really know in which category i should put this question however : Does anyone know what extension, library i can use in python which will control notifications appearing in windows sidebar?
exactly click on it when it appears in bottom right corner
dont really know if autopygui has this possibility
@thorny narwhal sounds like a #user-interfaces question
i posted this question in general, in 30secs i get answer. thanks!
One of cloud provider
I'm pretty new to python, and i was wondering some things about using Zoom API and stuff message me if you want to help
I've never used that before
Hello, about making automation scripts is right channel?
@heavy knot Can you help me? auto-py-to-exe
@midnight atlas I can
Hello, about making automation scripts is right channel?
@unkempt tendon sure
@pearl pawn this might be a better channel actually, it'll be quieter.
I'm also not sure how vim and neovim compare
I've been working on learning vim for the past few days
@rough marlin thanks
Comment indentation
Does anyone know how to fix indentation problem with comments?
are you pasting from somewhere?
I've seen that actually in neovim. What plugins (if any) are you using@pearl pawn
Fwiw, you don't have to use plug anymore
That's true but it doesn't make a difference at my level of vim usage
Anyone here do python development from an iPad?
do i need to do something extra for pycharm to detect a module in a subdirectory?
it keeps saying "can't find module" even with the correct source directories set
@midnight atlas I can
@copper meadow no need, it was my antivirus that was blocking it โ
yea, it was the same case with me too once @midnight atlas
๐
the argument "master" is not a argument in 'git rev-prase'@drowsy lava
@drowsy lava https://git-scm.com/docs/git-rev-parse
๐
oh
The 5000 changes and a repo branch that doesn't exist I can't remove them
try ping a helper
huh alright
@elder river hello there I wanna know how to fix all of this
I'm not sure how this happened
I just enabled the git extension and there came out
holi ****
that it a lot of files
yes and I have no idea what is happening
๐ฉ
nice pfp though
have to go
I am getting a very frustrating error with running docker:
duck@system:~$ docker-machine create -d virtualbox testdriven-dev
Running pre-create checks...
Error with pre-create check: "VBoxManage not found. Make sure VirtualBox is installed and VBoxManage is in the path"
I have already installed virtualbox and vboxmanage is on the PATH, I can run it on my own but docker can't.
I have already looked up docker forum and stackoverflow with no solution
duck@system:~$ which VBoxManage
/usr/bin/VBoxManage
duck@system:~$ which docker
/usr/bin/docker
Status: Solved
does git rebase <branch> put commits from the currently checked out branch onto the one specified in <>?
and how would I rebase from a specific hash?
You can specify a commit/commit range/branch after the target
Can someone ELI5 what Jupyter notebooks is and what it's used for? It looks sort of like a REPL session that you can also save to file and stuff. is that accurate? and if so, what's the use for it?
@signal crest Jupiter notebooks are generally used for scientific documentation and data science documentation, but I find them useful in teaching python to junior devs as well. As far as what it does, notebooks allow you to organise your code in a document like fashion, write documents just like you would in Word, but you are also able to write live code within the document. It's kind of hard to explain all in one go lol.
To be more ELI5 though, is a program to run code and write documents all in one.
Hm, I don't think I've seen it being able to embed word-like text before. that does seem kind of useful.
Not exactly word like, but the formula parsing gives it more of that feel
And is a whole lot easier than TeX lol
so basically it's purpose is mostly presentation of code rather than creation? that would explain my confusion because I always thought it was meant to help write code somehow
You can write code with it alone, but it's not as easy as a real editor or an ide
But if you need to write a technical paper, it's perfect lol
I do like some of the things that iPython brings though, but you don't have to run notebooks to get it
Is anyone active?
@gusty willow I'm ating you because you helped me the other time my bad. But do you know anything about tcp proxies. I need help understanding why my code works
Lol, usually the question is the other way around
As far as your question, yes, I know and understand proxies lol
why is it recieving the same data but printing different things
@gusty willow The website is sending Data to my proxy and all my proxy is doing is forwarding it to the client yet my client gets the entire content of the page while my proxy gets the permanant move address error
yet it works
idk y
This is the code
Problem is your proxy is seeing the initial request, but not following through with it. The client received the same message, but followed the redirect, and therefore, actually got the data back. @pearl pawn
Huh, ok I'll figure this redirect problem out then. Thanks bro!
Could it be because its http and not https?
hello everyone
i currently have a command line tool
and i want to make sure that a command doesn't work if a user hasn't run a command before
for example
package_name supercharge
and only if they run that command before, this should work
package_name superpower
@sly sleet where would be a save protected location? I need to store a bool variable so.. is there any easier way/
hm
there's not much you can do to prevent them from reading it
chmod/chown could help
but they could always use root
true... though, how can i store it into some kind of system variable?
like a quick access variable?
Writing somewhere in /etc will be root-protected
hi guys! I could not find anything like this online, so here is a class that allows you to run any command with input/output processed by python. supports commands that require user input:
https://github.com/vemel/awscliv2/blob/master/awscliv2/interactive_process.py
InteractiveProcess(["ls"]).run() # just run and exit
InteractiveProcess(["aws", "configure"]).run() # this one uses user input
Note that it does not use shell=True, so should be secure. Happy keylogging!
@thorn berry nice! what does it offer compared to pip?
no, its actually not a competitor to pip
you can install it using pip
@mortal panther it is for installing applications and packages into linux
omg, I see ๐ cleaner README is a must then
thanks for the feedback @mortal panther i will work on that
I've been spending 7-8 hours for the past 2-3 days trying to improve it, haven't gotten around to the docs and stuff lol
in fact, unlike other package managers, i am planning to add multi-threaded performance, so you can install packages at the same time
i believe apt uses multithreading
that i think it true however, you cannot run multiple instances of apt
anyway to install multiple packages of apt I will need to make sure that I write some complex code, to increase the speed
lol though i am kinda the only one working on this right now, so its going to be very hard for me to do so...
how do I point a Django docker image/container to a database running in another docker container? The Django one needs to access it at both build and run time
Build a network
the fact it's django probably doesn't matter. Just how can I set things up so that one docker thing can access a database running in a different docker thing
Why would access be needed during build?
That should be separate container
Depending on your needs you can rely on bridge network since they will be separate ports
Iโm on mobile so I canโt look at docker setup
I think I'm just gonna use a VPS on the internet for testing, which is all I need right now
and I know that that works and how that works
Is this a public repo?
no, and annoyingly I can't make it public either
Ok, then
This is a pretty temporary thing, I just want it done with
Well carry on, I was just going to peek and see
# Your settings should already be inside ./app/, or use the Docker --env--file option
# Include Django and Gunicorn in requirements.txt
FROM python:3
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
ENV DB_NAME=db \
DB_USER=docker \
DB_PASSWORD=docker \
DB_HOST="goddammit.just.work" \
DB_PORT=5432
RUN python manage.py collectstatic --noinput
RUN python manage.py migrate --noinput
EXPOSE 8000
CMD ["gunicorn", "--bind", ":8000", "--workers", "3", "brac.wsgi:application"]
version: '3.1'
services:
cheapestpintin_db:
image: postgres
restart: always
environment:
- POSTGRES_PASSWORD=docker
- POSTGRES_USER=docker
- POSTGRES_DB=db
# - PGDATA=/var/lib/postgresql/data/pgdata
container_name: cheapestpintin_db
ports:
- 5432:5432
# volumes:
# - ./volume/dbx:/var/lib/postgresql/data
That's the dockerfile for the app and the compose file I'm running the database with. Not all that interesting
the actual app is just generic boring CRUD django
no, the compose file is literally just the database - because in production the database will be on GCP
this is just me trying to figure out how to get the app ready for that
what approach do people take to ensure that they're up to date with master? Files like setup.cfg and stuff like that ๐ค is there a hook for that? Or do you just diff against master
@true vapor then just add it to the compose file, you don't have to leave it that way when you are ready for deployment, but that simplifies intra-container networking greatly
@topaz aspen that is why i use branches, so I don't have to worry about it. I only deal with sync problems when I'm read to merge
@gusty willow i have branches hang around for a while sometimes :/
I'll have a look at doing that - I treid a while ago and messed it up. But yeah, you're right, I should just figure it out
right ... so do i, but when i'm ready, i'll just merge in the final destination and deal with problems
I kinda forgot it was a thing
sometimes there are changes on master i want to bring in though, i'll just diff it's fine
@true vapor the reason you are probably having so many problems is the fact that one is running as a true container, and the other in a compose ... compose creates a new network isolated from the rest of the docker environment, so without a little hacking, it's hard to make the two talk ... so you need to do either 100% compose, or 100% not compose lol
you are making your own life harder doing it the way you are doing it
No - I meant having them both in the same compose file
but yeah, I'll put try putting them both in the compose file again
Yes, they should be in same compose file
When I get a moment, Iโll work on docker-compose
anyone know how to create a git ref for a github repo
Is there any way to make pycharm ignore "unreachable" code under an if __debug__ ?
what do you mean by a "git ref"? @verbal linden
Can someone explain what the difference in running the rdp file from a RSS web client and just rdping to the server ip is? The rdp file takes me into my vm pool but what about it does that?
version: '3.1'
services:
cheapestpintin_db:
image: postgres
restart: always
environment:
- POSTGRES_PASSWORD=docker
- POSTGRES_USER=docker
- POSTGRES_DB=db
# - PGDATA=/var/lib/postgresql/data/pgdata
container_name: cheapestpintin_db
ports:
- 5432:5432
network: djangonetwork
# volumes:
# - ./volume/dbx:/var/lib/postgresql/data
django:
build:
context: .
dockerfile: django.dockerfile
image: django
restart: always
networks: djangonetwork```
@true vapor
change db host to cheapestpintin_db or shorten the name
couldn't test since don't have repo but hopefully that helps
Hm - am I supposed to do something else?
services.django.networks contains an invalid type, it should be an array, or an object
Unsupported config option for services.cheapestpintin_db: 'network'
is what I get after doing docker-compose up
services.cheapestpintin_db.networks contains an invalid type, it should be an array, or an object
services.django.networks contains an invalid type, it should be an array, or an object
Do I need to change it to
networks:
- djangonetwork```
see if django can ping postgres
There's some problems with my application apparently - need to fix those first. But this is much further than I made it before
docker exec -it django ping <postgresimagename>
I prefer docker-compose build option, instead of trying to push a bunch of stuff into docker compose, just let it handle building and running
cool
you honestly saved me so much time and fuckery that would have driven me insane. I have no idea how much time you spent on this, but you saved me probably hours - thanks a ton for that. One day I'll (probably) take the time to learn the basics of this stuff properly, but thanks for the help
That's fine - I was mostly procrastinating with other stuff anyway, it was still super helpful
Hello. I made a git post-receive hook as a bash script to get the latest commit information and format it with a python script. Then the message generated will be sent to discord through a webhook post request.
My issue with this is that when I test the python script out with some data instead of the sys.argsv[] that get passed through the bash script, the message is sent correctly, in that it sends the two messages generated.
But when I use the hook itself and push something, it sends only the first message without the changelog. I split the messages to send multiple messages that fit in the discord message limit.
Here is a screenshot of how the message is being sent. The first messages are sent correctly with the header and the changelog text (modified readme) which are sent directly from the python script . The ones after are triggered through the shell script of the hook and are missing the second message with the modified readme line.
This is the code that sends the message:
def Send(self,messages):
count = 0
for message in messages:
count += 1
content = self.template.replace("$content",message)
requests.post(self.hook, data=content.encode('utf-8'), headers=self.type)
if len(messages) > count : time.sleep(1.5)
I don't have much experience with the requests library or networking in general. When debugging, it seemed that the message seems to be correct and is being sent, but it never arrives to discord and it doesn't throw out any error either.
hi
can any one help me with my code i don't understand why it dos not work ...
any help ?
i will be tankfull
@gaunt tundra go ahead and ask your question, dont "ask to ask"
we have a guide for asking good questions:
!ask
hm. where is the bot?
anyway... try to include your code (as text, not as a screenshot) and the full error output if you get an error. please also try to describe the output you wanted, and how it is different from what actually happened.
it's not a python error but the module that i used in the code dos not act like i want to ....
i will send the code just a sec
#!/usr/bin/env/ python
import pynput
from pynput.keyboard import Key, Listener
from pynput import keyboard
keys = []
#bad_keys = [Key.enter, Key.space, Key.alt_l, Key.esc]
def get_key_name(key):
if isinstance(key, keyboard.KeyCode):
return key.char
else:
return str(key)
def on_press(key):
keys.append(key)
write_file(keys)
def write_file(keys):
with open('log.txt', 'w') as f:
for key in keys:
k = str(key).replace("'", "")
f.write(k)
#f.write(' ')
def on_release(key):
if key == key.esc :
return false
with Listener(on_press = on_press ) as Listener:
Listener.join()
Listner.start()
controller = keyboard.controller()
I need help with something. I have a python program that schedules jobs using the sched module. The way i have the scheduler working right now is that it uses time.sleep() to wait until the time for the next job. Now I want to implement a way to get the status of the scheduler. So my question is how can I implement a way to ask for a status? The best would be if i could run "python program.py -status" and it would output something like "waiting until x time to start y task". First time on this discord let me know if i need to ask somewhere else. thanks
@sour burrow it sounds like you need 2 separate programs running: a job scheduling daemon to act as a server, and the client that lets you check the status and control the scheduler
otherwise you can have your program react to SIGINFO
or whatever that one is
yeah SIGINFO
you'd need some kind of protocol that communicates over a socket. its probably way too complicated for a simple application
in general if you have 2 programs that need to talk to each other, you need to use some kind of OS-level communication mechanism
which is why im suggesting maybe just using 1 single program instead
what about a pipe?
yeah i dont know
so you are thinking something like
python -m run_scheduler
then being able to run
python -m scheduler_status
interactively?
this is meant to be a daemon that stays running for a long time?
if thats possible, that would be what im looking for
yes. it schedules tasks for the next 24 hours, then calls itself so it loops
this is linux. rasperry pi to be more precise
i can show you my current code, but it will probably need to be change quite a bit if im gonna make it work interactively
i cant necessarily offer a solution but maybe someone can
ok. I'll look more into the server and client stuff. thank you
@sour burrow what about dumping the scheduler status into a file and then reading that
@everyone anyone interested in joining the TurboCharge package manager dev team please dm me
Please stop cross posting this
And surely, don't try to ping every person in the channel.
Lol
Am i understanding this correctly, that if we use the flag --use-feature=2020-resolver, and we use pip install -r reqs1.txt -r reqs2.txt will resolve dependencies differently then pip install -r reqs1.txt and pip install -r reqs2.txt seperately?
its more that the 2020 resolver uses a different algorithm that might produce different results from 2 separate invocations
i did a test run in our ci pipeline, where i installed reqs in 2 commands
now i repeated it in one, and it fails
caught me by suprise to say the least
how does it make sense that the outcome depends on the amount of commands though, should it?
is it really determenistic if it depends on the amount of commands
i mean, it probably is, if you use the same amount of commands, still, it doesn't make sense to me
maybe its still not resolving with currently-installed packages in mind?
good question. im sure the algorithm details are somewhere on github
autobuilds as a first time docker user
are a bit confusing
im a bit confused about the build rules
and the build context
build context == the scope of the build. So if you use . that means that the build process will process everything in the current directory. But if use a different context, then the COPY . . calls or whatever have different meaning. Lets say I set the build context to ./build/data, that means, when i use COPY or ADD that the "root" (or .) level directory that i can call from, is actually the hosts ./build/data. Generally, you don't see this a lot, but it is useful in certain situations.
build rules generally refer to limiting builds on certain git tags or branches
Is there a way to install only the dev dependencies through pipenv?
having a git error ```error: There was a problem with the editor 'vi'.
https://stackoverflow.com/questions/14607584/using-vim-for-git-commit-messages-broken-after-updating-janus does this help?
are you trying to use vi as your default editor? You may need to change your editor
@sand thistle
I did
yeah
it helped
but now I get a create mode 100644
which from what I'm reading is some meta data
what's odd is t of his was there before
are you getting an error?
haha, git status
i mean everything is fine
is there a way to compile a python file into a .exe/application
idk if this is the right help chat
@heavy knot pyinstaller is great for that
@sand thistle 100644 is the normal git mode
====================
So I've heard that pipenv and Pipfile are no longer the state of the art for python build and dependency management tools
I've heard of a pyproject.toml file which is supposedly compatible with multiple different build systems?
My question is, if I started a new python project right now, what build/dependency system should I use?
pipenv tbh
IMO pipenv isn't really great, but compared to other dependency managers, it is somewhat good
poetry could also be a valid choice, but I know some people who never managed to make it work
how do I make requirements.txt lol
like if I do pip freeze it gives me a list of all the packages
You redirect the output
ya i got that
pip freeze > requirements.txt
yeah ik that but it gives me all the packages i have
I mean, yeah, it will give all your packages in the current environment
the only packages I have installed for this particular project is pynput and docx
Did you installed them inside a venv?
can I manually make a requirements.txt then?
Yeah sure
It is actually recommended, so you let the packages set their dependencies themselves
like should will installing through requireme
ah yes I was wondering whether it will also download other things required by it
as I have seen requirements.txt of other people with random things
do I have to mention the version of each package?
Yeah, it is better if you do
ok cool
Thanks!
I'm not able to sign into github in vs code
I use manjaro Linux and my browser is Firefox
basically when I try to commit, it will tell me to sign in and I follow the process but it fails
I was able to reproduce the same thing after installing the Github Pull Request extension
basically it will take me to github. I sign in and press continue
poetry is about as neat as pipenv mainly because of pyproject.toml but then again it's missing quite a few things
I've yet to write a C/Rust extension for Python that's managed by poetry
when i click open with code-OSS url-handler, it takes me to Code and makes a new tab with file name as the token, and then nothing happens
so i follow the "Didn't work?" instructions
but when I paste the token and hit enter, I get sign in failed: Error: network error
I'm stumped
Currently this is my .travis.yml setup. Since Travis CI creates virtual environments by default, shouldn't I use poetry to install tox to the said environment, then invoke that instead?
install:
- pip install poetry coverage tox-travis
script:
- tox
after_success:
- bash <(curl -s https://codecov.io/bash)
Though I guess this avoids duplication from poetry installing dependencies at the top level and at the level of tox
I'm stumped
not anymore. finally signed in. I did open link with system handler this time. However that gave me another error (Writing login information to the keychain failed with error 'The name org.freedesktop.secrets was not provided by any .service files'). After installing qtkeychain and gnome-keyring, it finally signed in
Any recommendations on python specific or just general VSCode extentions?
Trying it out for the first time after using Pycharm a lot
so far got gitlens and rainbow brackets
Also, any extension that emulates Atom's Emmet autocomplete for html?
how does docker fit into a CI/CD pipeline
right now I have it set so that any pushes to my git repo, auto generate a new docker image
my goal is to then take that docker image and auto upload/run on my server
im using digital ocean, and I'm looking through their tuts right now
Hi, how do you guys work with venv in vscode? The problem I have is when I setup my vscode environment do use my .venv it tries to install pylint in it which is obviously not what I want.
Or is it correct to install the pylint in the venv?
It seems to be pylint is not something which should be in the requirements.txt, since it is not needed to run the project.
Yeah I know what it's for, but is it really the proper way to just remove it from the requierements? Installing pylint and mypy install additionall dependencies which bloat the requirements and are a hassle to remove
I mean, what else are you going to do?
keep them out of the requirements.txt @lost night
just install them seperately
it's ok to add additional packages to your venv
unless you regularly run pip freeze > requirements.txt which isn't the best idea
Got it
@lost night You could also add the development dependencies to a separate dev-requirements.txt file
you could add a first line in the dev-requirements.txt: -r requirements.txt that would include all the dependencies from the base requirements.txt file into your dev-requirements.txt
So I'm getting this error in VSCode and don't remember it messing it up last time I used it
C:\Users\sevsd\AppData\Local\Programs\Python\Python38-32\python.exe: can't open file 'Untitled-1': [Errno 2] No such file or directory```
I've just swapped over to PyCharm for now
I started getting into vim, and have found I like it enough that I'm wondering if there's a somewhat friendly CLI debugger option for python
how do pudb and other things beyond basic pdb stack up?
https://jdhao.github.io/2019/01/16/debug_python_in_terminal/ i've found this so far
but I'm wondering if there are any more obscure or recent tools that i've missed beyond those mentioned in that roundup.
https://github.com/rocky/python3-trepan/ i also saw this mentioned
So far, I've only used pycharm for debugging python
and I think GDB once in the distant past.
When should separate jobs be used in CI vs. just adding more steps? In my use cases, using separate jobs hasn't really felt justified, considering it requires spinning up a new agent and checking out code again.
Even if semantically it may make sense to have separate jobs
Like, a job to lint and a job to run tests.
One of my projects that on Travis with tox has the lint and test stages combined for each Python version
I didn't really mind running pre-commit twice instead of just once but running them on separate jobs seems a bit better as it decouples responsibilities a bit more
'm using Python 3.7.8, flask
ModuleNotFound: No module named 'SpeechRecognition' appears when I try to run my project.
But it works well when I try it with python -m speech_recognition:
still not working when I import speech_recognition
I'm not sure how smart VSCode's python autocompletion is supposed to be but does it complete
from pathlib import Path
for p in Path('.').rglob('*'):
p.```
for you? It should be rather trivial to infer the type of p here, yet I get no completion at all
I'd assume the culprit would be the fact that rglob doesn't actually specify its type
Was going to check the source, but my phone can't load 1.5k lines of code lol
Not quite VSCode, but since I use MPLS the behaviour is relatively the same, in this case, the language server was smart enough to infer that xs is a List[Path]
Where do I ask Git related question?
Here is fine 
In a Pull Request I submitted, the collaborator commented something
and I see the Resolve Conversation button
So, I agree with him in one comment and I disagree with him in other.
So, the comment which I agree, should I press Resolve Conversation?
and what about the disagree?
Whats the proper way to address this?
When you disagree, you'd usually comment on their comment
And usually (although some projects doesn't do that) when you applied a suggestion, committed and pushed it, you'd mark the conversation as resolved
I kind of disagree but I don't really mind, if they keep the changes as it just a grammatical related issue.
In one single commit, I made two changes. Of which one they disagree and one they agree.
So, in which I disagree but really dont mind, what should I comment exactly. I dont want to sound rude or bad.
The one which I agree, I will mark it as resolve conversation.
You should just explain why you are disagreeing with them
Disagreeing isn't a bad thing, it is a good thing actually
To maintain a good code base, you have to get thoughts from many people
The thing is its a grammatical issue. So they could be right too.
Although remember that they are the maintainers, so they have the final word
Well, that's up to you
Just explain your point of vue, why you think your wording is better
In another scenario, consider, I found I was wrong with the grammatical issue. Then what should I comment to sound proper and not too apologetic
I want to maintain etiquette
Tbh, if that's just a grammar issue, you don't need to apologize or whatever
It just happen, code reviews are here for that
And to make that disagreed change back to original, they will first have to Request Changes right?
I will ask them to request changes then?
They just commented, not requested changes?
Well, either way it is just a way of noting if you are favorable to the PR yet or not, it doesn't really mean anything
Yeah they just commented
Anyone can please let me know free or cracked software version of google map data crawler to extract data?
does anyone know how i can cd into a user folder in a batch file without stating the specific user? e.g %user%
You could use the %UserProfile% variable
~
Or ~ yeah
thanks
Hello, can I ask totally newbie questions here or is there a better place to do so ? (I would like to know if there's a search or filter option on https://pythondiscord.com/pages/resources/ or a way to show them all)
We're a large, friendly community focused around the Python programming language. Our community is open to those who wish to learn the language, as well as those looking to help others.
I made a user friendly audio steganography encoder and decoder in Google Colab
https://colab.research.google.com/drive/1IivCTimv8npSxfV2c8no9nhlBi78GVEf?usp=sharing
@sick smelt #community-meta would be a good place to bring up a discussion on that
@forest bay Thank you, I've realized that recentely ๐
However, because you made a PR from YOUR_ORGANIZATION_REPO instead of YOUR_MAIN_REPO, I'm unable to force push to your PR branch (even if you checked the "allow edits" box). Would you mind adding me to https://github.com/YOUR_ORGANIZATION_REPO/their_repo_name so i'm able to do that, and land this PR?
I sent a PR.
This was their reply.
So does this mean, I should add them as a collaborator?
to this repo https://github.com/YOUR_ORGANIZATION_REPO/their_repo_name?
Like going to Manage access -> and inviting them with write access?
Sounds like it
has anyone used GCP+django_storages? I'm trying to set my app up on GCP and there's practically no documentation for it, and I'm getting a weird error
you need a private key to sign credentials.the credentials you are currently using <class 'google.auth.compute_engine.credentials.Credentials'> just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.
This <link rel="stylesheet" href="{% static "public/css/materialize.css" %}"> is the line that's causing the issue, it works fine locally with a GCP credentials json thing
hey can anyone help me with pyinstaller?
@vapid lance that kinda question would be more suited in a general help channel (see #โ๏ฝhow-to-get-help)
How do I make it so that when I start up the docker image via docker-compose up it will update the files on the image? Right now the new files aren't being copied over, it just sticks with the files originally sent when first composing.
Dockerfile:docker FROM python:3.8-alpine WORKDIR . ENV FLASK_APP main.py ENV FLASK_RUN_HOST 0.0.0.0 RUN apk add --no-cache gcc musl-dev linux-headers COPY requirements.txt requirements.txt RUN pip install -r requirements.txt EXPOSE 5000 COPY . . CMD ["flask", "run"]docker-compose.yml:yml version: '3' services: web: build: . ports: - "5000:5000" redis: image: "redis:alpine"
You can either bind mount the source code (which mean the compose won't be portable anymore), or rebuild the image every time you launch compose using the --build flag
Thanks to caching, rebuilding will be almost instantaneous, but you can still optimize your image a bit
Moving EXPOSE and CMD just after your two ENV would make it really instantaneous
I wanted to make it eventually so that things will automatically be uploaded to the image as soon as they're edited (and saved), without me having to compose/build every time
I believe you have to use a volume or something for that but not gotten that far yet
Yeah, you could use a bind mount, but again it means that you will have to have a local copy of the code in order for the stack to work, which you'd probably not have in production
So would I not want to have a bind mount then?
Some prefers to rebuild, some prefers to have two compose, that's up to you
Flask doesn't auto-reload code like django does, right ?
Tbh I won't actually be using flask, just the tutorial I was following has that as example
I'm using redis for a discord bot in the long-term
Okay
Because if your app supports auto reload you'd want to use a bind mount, but if it doesn't that's really up to you
How would I know if it supports auto-reload or not?
I meant like django, if you modify the source code of your application it will reload the changes
But I'm 99% sure that d.py doesn't do that, does it?
If you bind mount a folder, any change that is done outside of the container will be done inside the container too, and vice versa
So it would allow you to hot reload the cog actually
So like when a file is uploaded itโll execute bot.reload_extension('cog')?
Right okay
Iโll do that then
Do you have a link to like a tutorial of how to do that? (Setup bind mount and watch for file changes)
Here is the docs the bind mounts https://docs.docker.com/compose/compose-file/#volumes
The docs for this are kinda bad though
What you'd have to do basically is firstly to put your app code inside a subdirectory
You could for example use WORKDIR /app before your first COPY
And then add a bind mount inside your compose using something like
volumes:
- .:/app```
And to watch for line changes you could use pywatchman, although I never tried that tbh https://pypi.org/project/pywatchman/
Alright, thanks ๐
Iโll check those out
Just to verify, I'd have the pywatchman on the docker image right? Cause files edited on my side would automatically be uploaded by the bind mound @warm pollen
Yup
By the way, Docker is creating an actual bind mount, it works the same way as you'd mount --bind any other folder
I've kinda come back round to my original problem now
I've changed the config file to add the mount, but I can't actually 'upload' the new structure of code
Would the easiest way to just be delete the current container and recreate it?
docker-compose up --build --no-cache
But I don't understand your problem though
Using just --build it should go through the build steps again
Right yea, I forgot the --build option
through sublime's build feature I am able to run the discord bot however there is no output
This is my build system file
Is it because I am using a custom build system?
the problem is fixed but I can't stop my bot from sublime
there are two instances of my bot open
@quaint saffron can you-- welp, did you solved it?
yea I had the FLASK_APP env as /code/main.py instead of code/main.py
Although I got another issue
The mount doesn't seem to be working
I make a change, refresh the webpage, and it doesn't update
version: '3'
services:
web:
build: .
ports:
- "5000:5000"
volumes:
- .:/code
redis:
image: "redis:alpine"```this is my yml file, so I have setup the mount
And I built the image with docker-compose --build
Hmm
If you kill and restart the container without building, will the web page change?
And it is uploaded the updated docker-compose.yml file to the container C:\Users\tizzy>docker exec -it redis_web_1 cat docker-compose.yml version: '3' services: web: build: . ports: - "5000:5000" volumes: - .:/code redis: image: "redis:alpine"
lemme try that rq
Restarting the container does update the code (using ctrl+c to close script and then docker-compose up)
But the mount part doesn't appear to be working
You restarted it without the --build option, right?
correct
Just docker-compose up
So it seems to be uploading the new file but not restarting?
Then the bind mount is working, my best bet is that flask is doing some kind of caching
I can't find anything about caching on the flask docs
Maybe it was your browser doing some caching?
Closing browser and reopening makes no difference
Nor does a hard-refresh (ctrl+f5)
And nor does using a completely different browser
So don't think it's the browser-end
I can confirm the file is definitely being sent to docker though, cause I can cat the file and it shows the new content
Just flask doesn't seem to be restarting, like it should (I know it's not restarting cause it'd show in logs if it was)
@warm pollen any ideas?
All I can really think of is using the pywatchman you linked earlier to restart the flask app
Do computers count as development tools for the purposes of this channel?
Right okay so now I need help with installing pywatchman. I'm not sure which container I need to install it onto @warm pollen
C:\Users\tizzy\PycharmProjects\redis>docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------
redis_redis_1 docker-entrypoint.sh redis ... Up 6379/tcp
redis_web_1 flask run Up 0.0.0.0:5000->5000/tcp```I've got two and not sure which to do it to.
redis_web_1, the python container
Right, okay
How do install homebrew onto the container? @warm pollen
Apparently I need to install the watchman thing onto the linux distro
for docker you use different package managers
I'm getting confused with how to use pywatchman because there's literally no docs
depending on what the base distro you're using for a container
There's some docs on the project homepage
It just links to watchman docs which links to another watchman docs thing
a debian/ubuntu based image will use apt, fedora-based ones will probably use rpm
homebrew is a mac version of the same idea as the linux package managers i mentioned above

https://facebook.github.io/watchman
@warm pollen this isn't for python though?
gtk, thanks @rough marlin ๐
If you go to the docs section on that page is says to install with brew install watchman @warm pollen
they mention building from source under the linux/mac section
You'd install the python binding through pypi here
Might as well look at the django source
Hold on
they also mention binaries here https://facebook.github.io/watchman/docs/install.html#binary-downloads-for-linux-macos-and-windows-beta
not sure if you need watchman inside a docker container, but which package manager you're using for each task will depend on where it's running and whether it's installing python packages or system ones
I don't know how to install those onto docker though, that's onto the machine I'm on @rough marlin
so, before we go any further
i think you should know a bit about how docker works
You're used to using a mac, right?
are you on mac right now?
Windows
ah, ok
Oh, I see.
ok, the documentation they have isn't very good.
I found it confusing too.
Never used linux stuff before which is why I'm struggling so much lol
That's ok.
