yeah we have several options to fix it listed here atm
#unix
grave jolt
the question is how much work are you willing to do? none of them are really that much effort
sage solar
so I downlaoded the latest iwlwifi packjage
it just contains a bunch of ucode files
anyone got any idea where those belong?
oh apparently just in /usr/firmware
grave jolt
that might just be firmware files then
sage solar
yup
i guess its worth a try then
so
wget https://wireless.wiki.kernel.org/_media/en/users/drivers/iwlwifi/iwlwifi-qu-48.13675109.0.tgz
tar xzf iwlwifi-qu-48.13675109.0.tgz
iwlwifi-qu-48.13675109.0.tgz
cd iwlwifi-Qu-48.13675109.0
sudp cp iwlwifi* /usr/lib/firmware
`
?
@grave jolt
grave jolt
if thats where the rest of your firmware goes then that seems correct
sage solar
just locate .ucode
grave jolt
looking at the release history for iwlwifi-backports, they did add some basic support changes for ax201 in june
lament crest
it's just /lib/firmware
grave jolt
but it has 2723 before that, but also calls its a 'preproduction' device
whatever that means
iwlwifi-backports is the actual kernel driver it looks like, just for clarificaiton
it seems pretty likely theyll have to compile the iwlwifi-backports driver too since it has a small code change for that device
looks like there is also a package available for 5.2 on https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.2.1/
which might be easier if the firmware itself isnt enough
though idk if they are even checking here anymore
lethal bronze
he;;p
hello do you know how to show hidden files extention
of files
and then rena,e it
rename it
@sage solar i already spended ton of effort and time
sage solar
you do ls -la in the directory you want to see your hidden files
lethal bronze
i mean extention of my files
like text.txt
is file creaated in text editor
@sage solar
sage solar
why would you want that?
lethal bronze
hello.exe
so i can rename
to py file
i want to make file.py
so i can open it in pycharm on ubuntu
lethal bronze
how
sage solar
or just make a new pycharm projkect and create a file there
warped nimbus
It's located in the current working directory of your terminal
lethal bronze
where is it
lethal bronze
i cant find f folder in my home
oh nvm
i see
wait
the file should be opening in pycharm
because its py file
lament crest
python files don't open in pycharm by default
lethal bronze
hoow do i make
py files to be open in pycharm
by deault
how do i change type file?
on ubuntu
lament crest
you would have to change the default application to pycharm. but this will also just open the file in whatever current project you have open. if you do not currntly have a project open, it will create a temporary project in /tmp
it is recommended that you create a new project in pycharm, and move the file to that project
lethal bronze
how do i change the type of the file
lament crest
change the type of file? do you mean change the extension, convert it to a different filetype, or do you mean change the default application for opening the file?
lament crest
mv myfile.txt myfile.py
that would be if you wanted to change myfile.txt to myfile.py
lethal bronze
how does it know location of myfile.txt
lament crest
you tell it
fallow gale
The directory you are currently in
lethal bronze
ok its in desktop for instance
fallow gale
Erm
lament crest
so if your file is in desktop, then you could cd ~/Desktop to get to your desktop, then type the above command.
fallow gale
Use terminal on your unix then do the above.
lament crest
just to check... are you about to try and change a .exe to a .py?
cus if so, that's not how it works.
i just realized you mentioned something a while ago about changing an .exe to .py
lament crest
okay
fallow gale
gotta
lethal bronze
yes
lethal bronze
file
lament crest
file.txt?
fallow gale
literally file
lament crest
or just file?
lament crest
what does the output of ls say?
lethal bronze
file
lethal bronze
type file is txt
lament crest
file type is not the same as file extension
fallow gale
It won't be if ls doesn't say .txt on it
lethal bronze
ok i renamed its extention to py
not working
same type as it was
plain text document
lament crest
🤔
lethal bronze
should be python
lethal bronze
when you right click on text file
and properties
it says file type
should be python
lament crest
can you show me what the output of ls -to shows?
lethal bronze
@f-Blade:~$ ls -to
total 56
drwxr-xr-x 6 f 4096 Jul 17 16:49 Desktop
-rw-r--r-- 1 f 0 Jul 17 16:44 file.py
drwxr-xr-x 3 f 4096 Jul 17 16:24 snap
drwxr-xr-x 3 f 4096 Jul 17 16:24 Documents
drwxr-xr-x 12 f 4096 Jul 17 15:29 backport-iwlwifi
drwxr-xr-x 2 f 4096 Jul 17 14:52 Downloads
drwxr-xr-x 22 f 4096 Jul 17 13:36 rtlwifi_new
drwxr-xr-x 2 f 4096 Jul 17 13:10 Music
drwxr-xr-x 2 f 4096 Jul 17 13:10 Pictures
drwxr-xr-x 2 f 4096 Jul 17 13:10 Public
drwxr-xr-x 2 f 4096 Jul 17 13:10 Templates
drwxr-xr-x 2 f 4096 Jul 17 13:10 Videos
-rw-r--r-- 1 f 8980 Jul 17 13:06 examples.desktop
f@f-Blade:~$
lament crest
well the file.py that's in that directory is an empty file.
and you change a files extension by renaming it.
lethal bronze
what do you mean empty file
lament crest
i mean the file file.py is empty
lethal bronze
yes
and
also my keys
razer synapse for ubuntu i can't download
okkkkkkkkkkkk'
file
@lament crest
so how do i create
the fileeeeeeeeeeeee
lament crest
...are you asking how to create a file?
main olive
@lethal bronze What's the problem?
lament crest
you can enable a setting. click File > Settings. go to the Editor group, and then the General section. the 2nd option should be "Change font size (Zoom) with Ctrl+Mouse Wheel"
click that. then in the editor, you just hold down control and use the mouse wheel, and it will, effectively, zoom in and out.
lethal bronze
HELLO
@main olive
@lament crest
do you know how to go back to previous code i just deleted in pycharm?
lament crest
hold Ctrl and press Z?
main olive
Same as most other programs, CTRL + Z
Also, no need to @ people - just ask.
It's seen as rude to @ people unless you were just having a conversation with them.
lethal bronze
hello
i bought wireless adapter
and its signal really
low
i think i can change its singal in netgear geniu software which is only working on windows because i use ubuntu
main olive
i want to be writable
bcs
stdin, stdout, stderr = ssh.exec_command(f'cat sb/users/robert.json')
m = json.load(stdout)
m["test"] = a
with stdout as f:
json.dump(m, f)
OSError: File not open for writing
neon burrow
You can't write to stdout.
That's the output of the command.
What do you want to accomplish here?
neon burrow
Ok, you want to write back to the same file?
main olive
yea
but is not only
`{"test": "the thing:"}
is
{"thing1": "do1", "thing2": "do2"}
is more then the test
neon burrow
Maybe you could do something like python ssh.exec_command(f'echo "{json.dumps(m, indent=4, sort_keys=True)}" > sb/users/robert.json')
When you want to write it back.
I didn't test that one, so I'm not sure if it's gonna work, but something along those lines
neon burrow
The former is not valid json, but I dunno.
main olive
yea it is wrong
oak shell
because it is evaluated in a shell as arguments to echo. That removes/evaluates the quotes.
you might have cat > sb/users/robert.json as your remote ssh comamnd and then write to its stdin.
main olive
cat {text} > sb/users.../?
torn snow
For the love of all things holy, please don't use f-strings with shell commands unless you call shlex.quote first
@main olive out of curiosity, have you ever heard of sshfs
If you're doing two-way writes over ssh, sshfs is a far cleaner solution
main olive
is first time using paramiko
main olive
ssh.exec_command(f'echo "{json.dumps(m, indent=4, sort_keys=True)}" > sb/users/robert.json')
there how can i add quotes?
main olive
`stdin, stdout, stderr = ssh.exec_command(f'cat sb/users/robert.json')
m = json.load(stdout)
if m["UUID"] == "a":
print("default")
f = stdout
m["UUID"] = str(a)
ssh.exec_command(f'echo "{json.dumps(m, indent=4, sort_keys=True)}" > sb/users/robert.json') `
like for that
i need to replace the stdout?
oak shell
no
drop all that
stdout of a process is that process' output stream, means you can only read from that.
to write data for another process to read, you would need to put it into its stdin channel, for example
but cat filename does not accept any input, it just reads from filename and outputs its contents.
also, using echo in any way here to output your complex data is terrible practice.
as you see yourself, quotes etc are breaking, and there will be more pitfalls. Just don't do that at all.
livid forge
what exactly are you trying to do, just get stdout and stderr from a shell command?
oak shell
save a file on a remote machine via ssh, I think
main olive
with stdin i get OSError: File is not open for reading
main olive
`stdin, stdout, stderr = ssh.exec_command(f'cat sb/users/robert.json')
m = json.load(stdout)
if m["UUID"] == "a":
print("default")
f = stdin
m["UUID"] = str(a)
json.dump(m, f)`
i open the file, i get the "uuid"i edit them and put it back
grave jolt
why not use paramiko's sftp api? it can directly open a file on the remote server for rw
main olive
edited
oak shell
the remote process will read from stdin and write to stdout, so you will have to write into its stdin and read from its stdout. But yes, use the sftp feature instead, as riff suggested and I linked earlier too.
livid forge
how were you planning on editing the file? full replacement of the file or something like sed? in either example i dont know why you would actually need to use stdin
unless i am misunderstanding.
main olive
f = ftp.open('sb/users/robert.json', 'w+') n = json.dump(f)
oak shell
I linked a stack overflow question earlier where one of the answers demonstrates the use of sftp
main olive
yea
ty
ill read now
n = json.load(f) File "C:\python\lib\json\__init__.py", line 296, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "C:\python\lib\json\__init__.py", line 348, in loads return _default_decoder.decode(s) File "C:\python\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\python\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
ftp = ssh.open_sftp()
f = ftp.open('sb/users/robert.json', 'w+')
n = json.load(f)
if n["UUID"] == "a":
print("default")
n["UUID"]=str(a)
json.dump(n)
f.close()```ok so i believe w+ will empty the file instantly
if you want to read+write i think you need to do r+
main olive
File "test.py", line 24, in <module> n = json.load(f) File "C:\python\lib\json\__init__.py", line 296, in load parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw) File "C:\python\lib\json\__init__.py", line 348, in loads return _default_decoder.decode(s) File "C:\python\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\python\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
oak shell
!codeblock
shy yokeBOT
main olive
uoops
my bad
f = ftp.open('sb/users/robert.json', 'r+')
n = json.load(f)
if n["UUID"] == "a":
print("default")
n["UUID"]=str(a)
json.dump(n, f)
f.close()
if i do that
it will append
not write
it need to be
w+
?
grave jolt
you need to seek back to the start after reading
the 'current position' is where writing occurs, and after reading the file thats the end of the file
grave jolt
if what you are writing is shorter than the current content you might also need to tell it to truncate the file to the new length
livid forge
but with w+ i believe youll empty the file
main olive
yea
ftp = ssh.open_sftp()
f = ftp.open('sb/users/robert.json', 'r+')
n = json.load(f)
if n["UUID"] == "a":
print("default")
n["UUID"]=str(a)
json.dump(n, f)
is appending
grave jolt
did you try what I said?
main olive
i didnt saw
let me read
and how can i do what did u say?
like
f = ftp.open('sb/users/robert.json', 'r')
d = ftp.open('sb/users/robert.json', 'w')
n = json.load(f)
if n["UUID"] == "a":
print("default")
with open() as b:
c = d
n["UUID"]=str(a)
json.dump(n, c)
```?open it once in the mode that allows both read and write
read the file
use the seek function to go back to the start
write the file
if needed, use truncate to change the file length to the length of the data you wrote
here are the docs on how to use those functions
main olive
what are offset and whence
i found
json.dump(n, f.seek(0))
File "C:\python\lib\json\__init__.py", line 180, in dump
fp.write(chunk)
AttributeError: 'NoneType' object has no attribute 'write'```means fp is None
main olive
and file is r+
oak shell
you mean f instead of fp?
main olive
i dont understand what do you mean
oak shell
what is fp
main olive
not defined
is nowhere
in script
script is
ftp = ssh.open_sftp()
f = ftp.open('sb/users/robert.json', 'r+')
n = json.load(f)
if n["UUID"] == "a":
print("default")
n["UUID"]=str(a)
json.dump(n, f.seek(0))
oh, wait. sorry
main olive
i thing i used seek wrong
livid forge
sorry yeah, looks like r+ wont truncate the file
oak shell
I mixed up the two lines, the fp name is internal in the json library. nvm that comment
oak shell
the problem is that you give f.seek(0) as file argument to json.dump
but the seek function returns nothing
you seek first to set the file cursor to the beginning again
then pass the original file object to json.dump
main olive
and that mean ill need to have 2 files?
oak shell
no
livid forge
something like for the write
f.seek(0)
json.dump(n,f)
f.truncate()
just f
livid forge
i think if you just use dump instead of f.write
it should work
since the filehandle still seeked to the start
main olive
size of f.truncate()?
oak shell
nothing, it defaults to your current cursor position.
livid forge
i edited, look now
main olive
f.truncate() TypeError: truncate() missing 1 required positional argument: 'size'
grave jolt
len(s) presumably
main olive
what is s?
livid forge
was just an example
grave jolt
whatever you write to the file
main olive
k
grave jolt
so you might not want to use json.dump to write to the file
instead dumps to a variable, then write that variable to the file yourself
that way you have the length of that data, and can pass it to trunicate
main olive
File "test.py", line 30, in <module> f.truncate(len(f)) TypeError: object of type 'SFTPFile' has no len()
should i delete the truncate?
grave jolt
correct, you need to get the size of the data that you are writing
no
len(f) attempts to get the length of the file, which isnt correct
oak shell
or use f.tell() to get the cursor position and use that as size
grave jolt
yeah that would prob work
livid forge
oh might be because its SFTPFile
grave jolt
i dont think files accept len
right you cant measure the size of the file anyway
because if you use the size of the file, then you arnt resizing the file
because its already that size
you need to size it to the size of the data, either using tell as mentioned above, or by measuring how much data you are writing by using dumps instead of dumping directly in to the file with dump
livid forge
it might also be easier to just open for read, close, open for write.
the example i gave will work on local filesystem but there may be hiccups using paramiko
since you will automatically empty the file on write
main olive
ftp = ssh.open_sftp()
f = ftp.open('sb/users/robert.json', 'r+')
n = json.load(f)
if n["UUID"] == "a":
print("default")
n["UUID"]=str(a)
f.seek(0)
json.dump(n, f)
``` it works goodwithout
truncate
livid forge
try writing a lot of data to the file and then writing a small amount of data to the file and see if it looks correct
if the file doesnt shrink there is an issue.
livid forge
write several hundred characters then, and then write again with the normal amount
and see if the file is correct.
main olive
ik is not correct
but for a file with i guess 30 lines
i edit
is ok?
30 characters**
livid forge
make the file larger then it normally is
then test
if the file shrinks to the expected size, then somewhere the truncation is occurring (which is what you want)
main olive
or how can i get the len of data?
livid forge
i mean for this test you can just eyeball it
main olive
yea it is ok
with 7 more arrg on json
is ok
and the "UUID" was last
everything is ok
livid forge
i doubt it
i just tested it
json.dump() does not truncate
unless paramiko is automtically doing that
somber stirrup
Are there any other popular drivers for touchpads besides Synaptics?
oak shell
libinput?
somber stirrup
Oh yeah, thanks
white solar
Microsoft precision
gritty stirrup
How do I allow my program to be installed and then called from anywhere>
Do i just have the install script add it to PATH?
docker-compose does it somehow... don't quite understand it tho
sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
oak shell
well yes, /usr/local/bin is on the default PATH
everything executable in there can be run directly as a command without specifying a full path
zinc bobcat
You can specify an entry point in your setup.py file: https://amir.rachum.com/blog/2017/07/28/python-entry-points/
oak shell
check echo "$PATH" to see all other locations on your specific PATH
or am I misunderstanding the question?
gritty stirrup
@oak shell My program isn't a single file, ideally I need something like a windows shortcut in /usr/local/bin
@zinc bobcat Its not a python program, thanks anyways
oak shell
wdym
not all of it needs to be in a PATH folder. Just the executable you need to launch, or a symlink to it
oak shell
make sure to secure your linked executables though (i.e. root ownership and not writable by others on the executable and all folders above it)
gritty stirrup
Can I ask why?
It has no power unless you know its built-in password anywasy
But I don't understand why you mentioned that explicitly
oak shell
if something on your PATH is world writable, unprivileged users can modify or replace it to trick a privileged user into running it
true, chances for an exploit like that are near zero on your personal computer, but worth to keep in mind for e.g. servers or public/shared computers
oak shell
for executables and directories usually 755, with owner root.
main olive
644 for non-executables
formal schooner
is there some kind of unix or linux equivalent of autohotkey
gritty stirrup
ahk works great for me
But tbh that's cause im, in the words of tom scott, a massive botcher
warped nimbus
It's fine for simple programs
I've just seen it arguably taken too far
Like why the hell can AHK even support GUIs in the first place, feels like it doesn't belong and it doesn't do an amazing job of it either
And people using curl.exe and win32 api calls to do stuff
bleh
vagrant fern
TIL that the program that Vim runs when you run :make can be customized via :compiler, and there is one built-in for pylint
The great thing about this is that if you have e.g. a flake8 compiler and run :make, vim will jump between the code based on its output
warped nimbus
blog posts can be a gold mine
I am grateful for the people that write those up
Would like to see this code jumping for myself
Will try
robust cave
ooh that was on reddit, I read it too
sleek compass
this might be a dumb question, but I want to set up anonymous ftp to let people download some datasets we've generated
it's actually all working well, but I have all write permissions denied on the folders (and have the owner set as an ftp user that I made)
but yeah, now even root can't move files in
so I have to change permissions every time I add new stuff? it seems like a bad way to do it, but I am unsure of what the proper config would be
warped nimbus
Take this with a grain of salt, I don't know what "proper" would be here
But you could create an ftp group and add users to that
nimble trail
I'm trying to build an rpm package with build tools using Python 2.7 (I know, not my choice), but when I run python setup.py bdist_rpm I get the error ERROR ambiguous python shebang in /usr/bin/myscript.py: #!/usr/bin/python. Change it to python3 (or python2) explicitly. As best I can tell, this is something setuptools itself is generating. It's not my code which has that shebang. How fix?
south star
sorry, not many ideas without knowing what generates the script, and didn't find a lot of references aside https://github.com/MariaDB/server/commit/2f368bb967#diff-69feedda5694553ac4e061c02a779c2cR222 and https://github.com/erocarrera/pefile/issues/65
livid forge
" but when I run python setup.py bdist_rpm I get the error "
do
python2 setup.py bdist_rpm
/usr/bin/python is generally symlinked to /usr/bin/python2.x, but i dont think redhat or fedora do this anymore
but if you want 3.x obviously just do ```bash
python3
shebangs are only used when executing directly from shell. if the file is passed directly to the python interpreter it's ignored.
if the script throwing the error is being called by something else down the line youll either need to change the shebang or alias python
sage solar
in fact they do, what rpm is mad about here is that it is never clear which python /usr/bin/python is pointing to and thus you should make clear what you are pointing to
livid forge
yeah i believe the warning is to stop people putting that in packages
nimble trail
This is solved (went with a different distribution solution in the end), but neither option mentioned above worked. Setting Python 2.7 as the shebang didn't work, using python2 specifically to build it didn't work (it was the only installed on the machine at the time anyway). What did sort of work was packaging it as Python 3, installing it, then manually going in and changing Python 3 to Python 2 in the shebang.
Also, it's not a warning, it stops packaging entirely. Very frustating day yesterday.
merry comet
hello guys ❤
I have a root folder with several folders and files and I need to use Python to rename all matching correspondences. For example, I want to rename files and folders that contain the word "test" and replace with "earth".
I used this code:
import os
def replace(fpath):
filenames = os.listdir()
for file in filenames:
os.rename(file, file.replace('test', 'earth'))
But nothing happens
livid forge
call the function
it works
replace('.')
$ ls -1
rename.py
test.txt
$ ./rename.py
$ ls -1
earth.txt
rename.py
just like this?
def replace(fpath):
filenames = os.listdir()
for file in filenames:
os.rename(file, file.replace('test', 'earth'))
replace('.')```no
your function is called 'replace'
you just defined it
you need to call it.
and in this example, this will only work on the top level directory
it wont affect contents of subdirectories
merry comet
It wont be like this? I'm calling the function 🙄
def replace(fpath):
filenames = os.listdir()
for file in filenames:
os.rename(file, file.replace('test', 'earth'))
replace('.')```
I don't know python, python is my only workaround at the momentyes like that
if you arent doing it as a project, then you can just use find xargs if youre on linux or using cygwin etc.
merry comet
xargs: cannot fit single argument within argument list size limit 😛
livid forge
how many files?
merry comet
A lot of them 😅
nimble trail
If you're actually just trying to rename some files, python myprog.py should be fine. Make sure you actually cd into the directory, though. This has the potential to fuck things up if you're not careful.
merry comet
The code above is working just need it goes through the sub folders 🤕
nimble trail
You have to cd into the subfolders.
Otherwise you're just renaming everything in the root folder.
merry comet
I know but the purpose of script would be to it go automatically through all subfolders otherwise I could just use rename command
nimble trail
Right, but what I mean is you have to use something like os.walk().
In your original script, you're not changing folders, you're just replace the same one over and over.
merry comet
Ahh ok, sorry. The thing is that I don't have any knowledge in Python 😅
nimble trail
No problem. I guess I wasn't clear that I meant "change directory" from inside the program. (Or list all subfolders and do absolute paths)
merry comet
I don't know anymore what I'm doing
```import os
def replace(fpath):
for (fpath, dnames, fnames) in os.walk(fpath):
for f in fnames:
os.chdir(fpath)
if f.startswith('test'):
os.rename(f, f.replace('test', 'earth'))
replace('.')```
If someone can guide me or give a touch on the code I’ll really appreciate
File "rename.py", line 12, in <module>
replace('.')
File "rename.py", line 9, in replace
os.rename(f, f.replace('test', 'earth'))
FileNotFoundError: [Errno 2] No such file or directory: 'testimonial.php' -> 'earthimonial.php'```I don't think you need os.chdir if you already have os.walk, but I could be wrong.
merry comet
I don't have any clue
winter obsidian
Hey folks
I got a question about scripting
I have like 5 different python apps, all of them do some web scraping and runs every 3 min
I run them on my raspberry pi
Here is my question
I want to use tmux and write a script, it will create a new session, run each python program on new tabs and rename these tabs
Is there any good source of information you guys can recommend checking out?
Im doing the same process manually right now and it really takes some time
Opening a session, running all these 5 program
Its also a problem when i reboot my pi so i want to write a script
vestal turret
Any particular reason they should run in tmux?
winter obsidian
All these programs runs every 3 min. So thats why i thought using tmux would be better for that case
Is it possible to run them like on different terminals?
vestal turret
tmux has command line arguments that allow you to write a script.
You could also create a systemd job for them.
winter obsidian
So i can do all the stuff i manually do through scripting also right?
vestal turret
Yes.
simple phoenix
i started a python script with setsid, how can i kill it i need to update it
winter obsidian
Alright i gotta do some research then. Thanks!
merry comet
Find the service id and then kill it
simple phoenix
how do i do that
vestal turret
How did you start the script exactly?
winter obsidian
#!/bin/bash
tmux new -s scripts -d
#Earhquake script
tmux rename-window earthq
cd /home/pi/'Git Repos'/Earthquake-Reminder
python3 Main.py 2
#Film reminder script
tmux new-window -n film
cd /home/pi/'Git Repos'/Film-Reminder
python3 Main.py 2
#News reminder script
tmux new-window -n news
cd /home/pi/'Git Repos'/News-Reminder
python3 Main.py 2
#Eksi reminder
tmux new-window -n eksi
cd /home/pi/'Git Repos'/Eksi-Seyler
python3 Main.py 2
#Ghacks reminder
tmux new-window -n ghacks
cd /home/pi/'Git Repos'/GhacksNet-Reminder
python3 Main.py 2
#Oyungezer reminder
tmux new-window -n oyungezer
cd /home/pi/'Git Repos'/Oyungezer
python3 Main.py 2
tmux attach-session -t scripts
I guess im doing something wrong
Everything is fine except it runs the code on main terminal, not inside tmux. What im doing wrong?
Btw what im trying to do is i want to write a script that will run my 6 different python program, which they will be executed every 3 min (time.sleep(180) at the end of each program)
But the script above ran them on terminal, not inside tmux itself
vestal turret
The new window command allows you to supply a command.
winter obsidian
Alrightttt
I did it!!
#!/bin/bash
tmux new -s scripts -d
#Earhquake script
tmux rename-window earthq
tmux send-keys $earthq 'cd /home/pi/"Git Repos"/Earthquake-Reminder' ENTER
tmux send-keys $earthq 'python3 Main.py 2' ENTER
#Film reminder script
tmux new-window -n film
tmux select-pane -t film
tmux send-keys $film 'cd /home/pi/"Git Repos"/Film-Reminder' ENTER
tmux send-keys $film 'python3 Main.py 2' ENTER
#News reminder script
tmux new-window -n news
tmux select-pane -t news
tmux send-keys $news 'cd /home/pi/"Git Repos"/News-Reminder' ENTER
tmux send-keys $news 'python3 Main.py 2' ENTER
#Eksi reminder
tmux new-window -n eksi
tmux select-pane -t eksi
tmux send-keys $eksi 'cd /home/pi/"Git Repos"/Eksi-Seyler' ENTER
tmux send-keys $eksi 'python3 Main.py 2' ENTER
#Ghacks reminder
tmux new-window -n ghacks
tmux select-pane -t ghacks
tmux send-keys $ghacks 'cd /home/pi/"Git Repos"/GhacksNet-Reminder' ENTER
tmux send-keys $ghacks 'python3 Main.py 2' ENTER
#Oyungezer reminder
tmux new-window -n oyungezer
tmux select-pane -t oyungezer
tmux send-keys $oyungezer 'cd /home/pi/"Git Repos"/Oyungezer' ENTER
tmux send-keys $oyungezer 'python3 Main.py 2' ENTER
tmux attach-session -t scripts
Idk if thats a proper way to do that but it works 😅
winter obsidian
I got one last question
The code above works fine, but it doesnt go back to main terminal. I mean tmux stays opened. I want to attach that session and close the tmux page
How can i do that?
white solar
exit?
winter obsidian
Found this. Here it says when creating a session the -d switch prevents it loading in ur terminal
#!/bin/bash
tmux new-session -d -s scripts
#Earhquake script
tmux rename-window earthq
tmux send-keys 'cd /home/pi/"Git Repos"/Earthquake-Reminder' ENTER
tmux send-keys 'python3 Main.py 2' ENTER
#Film reminder script
tmux new-window -n film
tmux select-pane -t film
tmux send-keys 'cd /home/pi/"Git Repos"/Film-Reminder' ENTER
tmux send-keys 'python3 Main.py 2' ENTER
#News reminder script
tmux new-window -n news
tmux select-pane -t news
tmux send-keys 'cd /home/pi/"Git Repos"/News-Reminder' ENTER
tmux send-keys 'python3 Main.py 2' ENTER
#Eksi reminder
tmux new-window -n eksi
tmux select-pane -t eksi
tmux send-keys 'cd /home/pi/"Git Repos"/Eksi-Seyler' ENTER
tmux send-keys 'python3 Main.py 2' ENTER
#Ghacks reminder
tmux new-window -n ghacks
tmux select-pane -t ghacks
tmux send-keys 'cd /home/pi/"Git Repos"/GhacksNet-Reminder' ENTER
tmux send-keys 'python3 Main.py 2' ENTER
#Oyungezer reminder
tmux new-window -n oyungezer
tmux select-pane -t oyungezer
tmux send-keys 'cd /home/pi/"Git Repos"/Oyungezer' ENTER
tmux send-keys 'python3 Main.py 2' ENTER
tmux select-window -t earthq
tmux -2 attach-session -t scripts
@white solar tried to put exit at the end of script but didnt work
Basicly i want to prevent tmux session opening in my terminal
Oh im a fool
tmux -2 attach-session -t scripts that command, brings tmux session to the foreground
Yay it works now!
formal schooner
any nginx heads around?
root /var/www/foo;
index index.html;
location ~ /foo(/foo)*/? {
try_files index.html =404;
}
location / {
try_files $uri $uri/ =404;
}
this works when i navigate to / by showing me the index.html as expected, but when i try to navigate to /foo i get 404 instead of also seeing the index.html there
formal schooner
update: i had to write /index.html not index.html
stoic hornet
Looking for a bit of help:
I'm running CentOS 7 and have a bash script that basically opens a screen and runs a python script in it. When the python script stops the screen closes, I'd like for the screen to stay open.
in the bash script:
screen -S test -dm python3.7 test.py
does anyone know how I get the screen to stay open after the test script finishes/crashes etc
gritty stirrup
@formal schooner Sorry man, thought you wouldn't mind.
I have a bit of software that aims to be a "beginner" version deployment thing, and one of its uses is replacing screen.
Would I be able to recommend it in cases like this?
formal schooner
replacing GNU screen?
im not sure what you mean by beginner version deployment
@stoic hornet you can create a new screen and then send a command after
screen -S test -dm sh
screen -S test -X python3.7 test.py
stoic hornet
ah
formal schooner
where maybe youd use bash instead of sh depending on what shell you want
stoic hornet
thanks very much
formal schooner
im a tmux guy myself so i know very little about screen
(mostly cause i learned tmux first, no real preference)
gritty stirrup
@formal schooner you would sent up a config file with your command, run sharpops. And it'll run in its own "screen" automatically
With logging etc etc
formal schooner
hm. writing your own terminal multiplexer is not an easy task i imagine
what about wrapping screen or tmux instead?
there are many "tmux managers" out there
gritty stirrup
Well I've nearly built the software
It's actual purpose ain't to replace screen and tmux, it's just sorta ended up being a side effect
formal schooner
i mean, yeah you're welcome to recommend your own tools here
i thought you were asking if you should start working on it 😃
formal schooner
i assume youve seen tmuxp and the like?
formal schooner
im actually curious about this tool. you called it sharpops?
gritty stirrup
Yep.
It's on my GitHub...
Lemme go get it
It has a local server, that'll run all the commands for you.
The next plan is to allow it to be external by using ngixn
So you can access it through services like CircleCI
So you can have a full CI/CD project.
formal schooner
interesting. whats the use case for this?
gritty stirrup
Well... mostly me
formal schooner
looks like fabric
formal schooner
gritty stirrup
I don't want to use SSH.
Ok let's say I have a fresh server. I'll install docker and sharpops
I'll deploy my project, say my website, mcaq.me to circleci
It will run a sharpops image, and when it runs..
It'll all auto deploy
No more work required
formal schooner
a sharpops image is a list of operations for the server to run? with some templating involved
gritty stirrup
Ah ok sorry.
I'm gonna pre-install sharpops in a docker image
For my deployment config to use.
On circleCI
It also supports git polling
formal schooner
hm. im still not sure i understand, i might need a specific example
gritty stirrup
Which means it checks for your git to be updating,
When it is, it'll git pull and run.
formal schooner
if you think other people could benefit from the tool maybe you could make an example/ dir and demonstrate how it all works there
cause im still not following, sorry
gritty stirrup
All my projects will be using it soon-ish
But yeah, a examples dir will be 100% included
Cause I think their are lots of uses for this
formal schooner
cool. that will help i think
gritty stirrup
I want to get them all down
formal schooner
what is the basic model though
whats a sharpops image, something like a templated shell script that runs things concurrently?
gritty stirrup
Oh ok
Maybe that was a bad example
Try and start from square one again
So sharpops will come installed as a executable.
On Linux, it'll create a "sharpops server" for you as a Linux service.
So you don't need to worry about that
Say you want your "cool new discord bot" to use new code as it's added to GitHub, and run on a Linux reboot
You would add all that to a sharpops config file in your project, and a bit like docker-compose, run sharpops
It'll read the config file and execute everything in it, after it's been sent to the server
formal schooner
- ssh into server, install sharpops and enable/start the service with systemct or whatever service manager i'm using
- scp the sharpops config file and run it with sharpops, creating a sharpops "project"
- sharpops will then do whatever the config file tells it to do
is that right?
and sharpops config i assume is declarative and re-entrant, rather than me writing a 1000 line shell script to do all that stuff manually
gritty stirrup
sharpops installs the service system with systemctl for you
Yes. It's a config ini
formal schooner
sure, yeah, or your package manager might do that
gritty stirrup
Except from that, yes.
You would include the sharpops config file in the GitHub repo ideally.
I'm gonna have a system for making sure it reads the latest config.
gritty stirrup
Well what I mean is.
Say sharpops is active for "my cool discord bot"
And it's waiting for master branch to update
If it does, and it has changed to the sharpops config
It'll try and change to use that one
You'll be able to disable that feature of course
formal schooner
how would that work practically?
scp sharpops.ini admin@my-server:/var/discord-bot/sharpops.ini
ssh admin@my-server 'sharpops run /var/discord-bot/sharpops.ini
something like that?
gritty stirrup
Earlier, I was talking about my "advanced usage"
formal schooner
but the point is that if it sees the config file has changed, it will auto-update itself
gritty stirrup
Yeah
formal schooner
eg does it know that it can't deploy until the test suite finishes?
formal schooner
and how do you communicate w/ the server if not ssh? http? custom protocol?
gritty stirrup
So.
I'm gonna let you expose the sharpops server through Linux
So you can send requests to my server
That's what I meant by the sharpops image
So you don't need to install it in your CI
Just download the image, clone the git repo
Make the sharpops server "external"
(I'll add a option)
formal schooner
so sharpops can either run commands locally or send commands to another server via ssh
formal schooner
i dont know how CI servers typically work
formal schooner
how would sharpops running on Box A send commands to my CI server which doesnt have sharpops on it?
gritty stirrup
If you host it externally, (I.e expose the point")
Sharpops will force you to make a "allowed commands" and "allowed repos" list
So only you can control it
formal schooner
you dont have to sit here explaining it to me btw
gritty stirrup
No this is great
formal schooner
(but it might be good to get this explanation tightened up if youre going to share it)
gritty stirrup
So CI is all about testing and stuff
formal schooner
yeah i know that
back up a sec though
i have 3 machines:
- my laptop
- my bot server
- a CI server
i install sharpops on the bot server
gritty stirrup
Ok so
Do you want beginner setup, intermediate setup, or advanced setup?
That's the three "levels" sharpops covers
formal schooner
just practically how does sharpops work
gritty stirrup
That's not a easy question.
formal schooner
i still dont have a clear sense of what it is or does
gritty stirrup
Ok so
Right down at the bone
You send a http request to the sharpops server
It contains a bunch of names, but also a command
It's executed in a separate thread
formal schooner
this is how i might explain MPD for instance:
- it's a daemon
- it looks for music and playlist files in a directory that you configure
- it plays the music through each of several outputs that you configure, including ALSA, Pulseaudio, an HTTP stream, and others
gritty stirrup
I don't know how to explain it more than that
You send a command, it's run in its own thread
gritty stirrup
It's just that concept builds lots of options
Some I just didn't consider
Yes.
I want to call it a dameon... I wasnt sure it was
formal schooner
so how does the config file come into play?
yeah a daemon is just a program running in the background
gritty stirrup
Config file is so you don't need to do any curl nonsense
Sharpops handles all that Bullshit for you
One command installs for both windows and Linux
formal schooner
ok, so it reads and executes commands from a declarative config file, and/or you can send one-off command requests over HTTP
gritty stirrup
Yeah.
Well. no
You will always ideally use the config file.
But there's no reason you can't use curl etc
formal schooner
can you manage the sharpops server itself over http? like you can tear down or start up a new project, or upload a new config file?
that might be an additional use case for the HTTP thing
gritty stirrup
Security is my number 1 concern
formal schooner
yeah for sure. i assume the best thing to do would be to reverse-proxy through nginx or something?
if you want security
gritty stirrup
Im planning on adding run/stop options
Yeah.
Theres no reason to secure it for local connections
But external ones definitely
Stop options are hard because that means golang thread management
formal schooner
so now my question is.. how would this work on a CI server? you would put sharpops in a docker container, put the docker container on the CI server, run the container, then have the sharpops daemon on your bot machine send commands to the sharpops daemon in the container on the CI server?
gritty stirrup
Ok so.
One of the options in the config file will be to set a external sharpops dameon
So when you have sharpops in your CI
You would either just wget the sharpops file
Or clone the whole repo
Run sharpops run
Just like it was a normal server
It won't be treated any different
Config + sharpops will handle it all
formal schooner
but to be clear, you'd do sharpops run on your local machine, and since you set an external runner it would run the commands on the CI server?
conceptually it sounds like you have 2 parts:
- a client, which reads commands from a config file and sends them over HTTP
- a server, which receives commands over HTTP and runs them concurrently
even if they're physically the same binary
is that right?
formal schooner
...have you considered making them 2 separate binaries?
gritty stirrup
Yeah
formal schooner
not sure if that's smart tbh
gritty stirrup
I will seperate them
They are together for early development
It's a pain to get 2 working
formal schooner
sharpops run --server-only and sharpops run --client-only could do the trick
gritty stirrup
Right now it's just sharpops --server
But if you ever have to run that command, your using it wrong
Sharpops dameon should always handle itself
formal schooner
yeah it would be run by your system's service manager
gritty stirrup
Exactly
formal schooner
but the point is on the CI server i wouldnt run the client server
and on my local i might not want a server except for testing
so on my dev machine i might do
sharpops --server --debug &
sharpops run
and the docker container might have
sharpops --client
In the CI server you only need the client
You shouldn't need debug, itll go to the logs
sharpops --logs
Like docker
formal schooner
cool
gritty stirrup
You should never mess with the docker daemeon
formal schooner
yeah this seems useful... almost like a "non global" systemd?
formal schooner
like imagine if systemctl or s6 or runit supported "projects"
gritty stirrup
But its uses are pretty huge
gritty stirrup
It's good for me
formal schooner
if the dependency management is good it could replace makefiles...
gritty stirrup
Cause when it's done, I'm gonna have heroku for my RPI
Lol
What do you mean "dependency manager"
Pipenv support etc?
formal schooner
again, like i need to run the full test suite before i deploy
or i need to clone from master, run tests, and copy to /var/bot in that order
or whatever you might do
gritty stirrup
Mmm.
I didn't think of anyone doing local tests
I might add command order,m
You can define more than 1 command per config
formal schooner
i see, so it doesnt handle dependencies between commands
gritty stirrup
Dependencies yes, very difficult
Go look at any of my projects
They all use it
.sharpops.ini
By dependencies... what would you want me to support?
pipenv, etc?
I have no idea what it means
formal schooner
i mean dependencies between tasks
gritty stirrup
Explain that a bit.
formal schooner
lets say the git clone fails
i dont want to run anything after that
and any other operations i might run should always wait for the git clone to finish
so all my other tasks "depend on" git clone
worn apex
is this a shell script?
formal schooner
@worn apex they developed a tool called sharp-ops
gritty stirrup
This is complicated and I have no idea what it is right now
worn apex
oh
formal schooner
and we're working out how to explain it to people
gritty stirrup
Its sorta "evolving"
formal schooner
lets see if i can get the explanation down 😃
gritty stirrup
Hey if you guys know golang, we can work on it hahah
Salt so yeah, I suppose the issue there is how much nesting I need to support
formal schooner
it depends on the use cases you want to support
maybe v1 has no support at all for it
and v2 will support it
gritty stirrup
Likely a option.
For beginners, it's meant to run to keep your service alive
You see a lot of beginners that want that
There's nothing for it
Especially keeping it alive + git polling
For advanced users, everything should be dockerized
Or at least, that's what I do.
So dependencies are pretty useless for me.
I have none.
I think it's the intermediate users that need dependencies.
Does that make sense?
The people in the middle?
Of those two groups?
formal schooner
maybe. i might be envisioning broader use cases too
you could use this to provision a whole server
instead of ansible
formal schooner
you bootstrap by installing sharpops and it has a server-wide config
gritty stirrup
So we need to figure out how to make dependencies easier...
Are you interested in joining me?
You seem interested.
formal schooner
heck, when the config completes successfully it should shut off its own http server, for security
formal schooner
i mean if youre provisioning a server
gritty stirrup
Wdym?
gritty stirrup
Oh. running once is built in
Like pipenv.
You can configure scripts
That actually read .env :D
So it's pipenv... but for anything
formal schooner
yeah that makes sense
formal schooner
like i said, replace make 😛
formal schooner
@worn apex this is my attempt at explanation:
sharpops is a local command runner, with a client/server architecture. The server runs commands within a "project", like polling continuously for a new push to a git branch and then cloning the latest commit. The server receives these commands over HTTP. The client reads commands from a config file and sends them to the server.
does that look right to you @gritty stirrup ?
gritty stirrup
Yep.
Git isn't the only option, but it's the "beginners" option
Because it's super simple
formal schooner
yeah for sure
gritty stirrup
I quite like the one command installs
formal schooner
the fact that you can "chain" these is interesting
gritty stirrup
But Jesus : the windows one was a nightmare
wdym chain?
Like multiple commands,m
?
I call them "runners"
You are sending "runner requests" to the "sharpops daemeon"
formal schooner
why not just command?
i would say that the daemon itself is the runner
the runner runs commands
gritty stirrup
Well not all "runners" are technically "commands"
For example, the one that does the git polling
formal schooner
how about "tasks" then
formal schooner
chaining example: i have 10 servers on a cluster, and 1 machine to act as a head node. i can run sharpops on my laptop, which tells the head node to upgrade each of the worker servers
instead of me connecting directly to each of the workers from my laptop
gritty stirrup
I mean yeah. But that's a shit tonne of config.
formal schooner
i imagine its true any other way you do it. there are probably also 100 other tools for managing HPC clusters
gritty stirrup
Yeah. I'm gonna implement all the features I had planned for v1
With a "security" idea in plan
Dependencies for v2
And see what happens from there
It's a surprisingly small amount of code rn
formal schooner
i think if you can get the explanation down, plus some examples. that will be a great start
gritty stirrup
Like 500 lines?
formal schooner
yeah its a very simple design
gritty stirrup
Simple design, lots of possibilities
gritty stirrup
Yep.
formal schooner
laptop ----> head node ----> worker1
|---> worker2
|---> worker3
this is what i was suggesting for the cluster btw
gritty stirrup
So each worker is running sharpops
And the head node runs sharpops in sharpops?
🤔
formal schooner
why not 😛
formal schooner
again maybe this isnt useful for a cluster as such
but for example we had a bunch of production machines at my last company
gritty stirrup
Maybe that's a feature ideav sharpops-cluster
formal schooner
deployment was a pain in the ass
formal schooner
i think the simple way to do it is, the server can choose to forward commands to another client
instead of running them
but again until there's a practical use case for it, dont bother
this is just me thinking aloud
feel free to use the explanation that i gave for Random btw
maybe you can ask around on e.g. the sysadmin subreddit
or i guess for beginners it helps a lot too
as long as its not buggy i think this could be a potentially very nice tool. i dont have a ton of use case for it myself but
gritty stirrup
@formal schooner Sorry, i was gone.
That's my website using sharpops now.
I just need to add it to the CI
VERSION = 1
ENVFILE = .env
[runners]
name = "McA Designs Runners"
[runners.script]
name = "Run Docker"
command = "docker-compose down && docker-compose pull && docker-compose up -d"
I will likely make a "docker" task
That will have all that bash written into it
formal schooner
you can have multiple [runners.script] sections?
gritty stirrup
Yep.
Then [scripts] is the pipfile like stuff
Anything in .env is passed in too.
formal schooner
have you considered putting the name in the ini header?
like how Git does it
so you don't have a bunch of repeated headers
[runners.script "Run Docker"]
command = "docker-compose down && docker-compose pull && docker-compose up -d"
I'm using a really well made Golang Ini module
And I don't think it supports that
marble bluff
on linux, how do you run a command with lower privileges in python?
changing the user requires root priveleges so i can't automate it
marble bluff
Alcorro
torn snow
@marble bluff what's the context for this?
marble bluff
uh
run user code like a grading machine
so it'll also need to be time limited so i can stop it later, i will also run with cpulimit or something
and i need to limit memory
torn snow
But why do you need to switch users
marble bluff
because the other user has lower privileges
so it can't touch any of the other files
vestal turret
Does su not work?
torn snow
Okay so is this like a gui or a service
marble bluff
that requires root priveleges and i don't think you can like automate it
service kidna
kinda
torn snow
Is it that e.g. the teacher opens it and it switches to a user account?
torn snow
Like what user is going to be opening it at first that would require you to switch users
marble bluff
and then you input your code in the textbox and it checks if your code works and if it successfully passes testcases
torn snow
...this is a website?
marble bluff
ye
torn snow
Wouldn't user stuff and all be in the browser then
torn snow
OH
Okay so
The user can give some command to be run and the server needs to run it right
marble bluff
yea but its a file
torn snow
For that you need a flat out sandbox, not just user switching
marble bluff
why?
torn snow
There's tons of things that you could still do it you switched users
E.g. eat up all your memory
marble bluff
yeah but i can limit the memory and time and cpu usage
as long as it doesn't touch any other files
vestal turret
su can also run a command as another user.
torn snow
No please don't do this, it's a really bad idea
Check out bubblewrap
It's a lightweight sandbox for Linux
Flatpak is built on it
Or firejail
vestal turret
Just like you would with a regular login.
But running arbitrary code can be quite dangerous.
torn snow
Or nsjail
Yeah
You want a proper sandbox around this
Not user switching
There are dozens of potentially fatal syscalls
marble bluff
okay
but i feel like it should be fine
it checks for importing
and it can only import math, so it cant run os commands
torn snow
Not good enough
I've written stuff before that can import modules you blacklisted
Python is really really hard to sandbox
vestal turret
You can look at how @shy yoke does it.
marble bluff
also, i have to do it for c++ and java as well
vestal turret
iirc it uses nsjail
torn snow
Also here's an article that shows how you might use nsjail to sandbox https://offbyinfinity.com/2017/12/sandboxing-imagemagick-with-nsjail/
Haha yeah nsjail is good at this
marble bluff
okay interesting
thanks for the help!
how scalable is using bubblewrap, nsjail, etc tho?
like can i run multiple commands on it
or does it duplicate every single time
marble bluff
can i test dangerous things in the python bot?