#unix
1 messages ยท Page 2 of 1
ah i got it in reverse
grep -P "[\x00-\x1F\x7F-\xFF]" input_file | od -x /dev/stdin should work based on this https://stackoverflow.com/questions/25720471/how-to-find-non-printable-characters-in-the-file and https://manpages.debian.org/jessie/coreutils/od.1
what about this
this is hex bytes right
is it possible to convert this to string
apparently xxd -r -p
question asks you to find sequence of non printable non ASCII characters and then print it in HEX
so this should be it
or rather it is not it
because you do something strange with sort and uniq for some reason?
just do grep '[^[:ascii:]]' -o | xxd -p /dev/stdin ie find all non asci characters and conver them to HEX using xxd without printing line numbers
I'm getting this weird error when I try to update openssh on my rhel server. Any ideas on the cause?
โ find /var/cache/dnf/ -name '*.rpm'
/var/cache/dnf/rhel-8-for-x86_64-baseos-rpms-ce63dfa6a7abcf7b/packages/openssh-server-8.0p1-13.el8.x86_64.rpm
โ sudo find /var/cache/dnf/ -name '*.rpm' -exec rpm -U {} \;
error: unpacking of archive failed on file /etc/ssh/sshd_config: cpio: utime failed - Resource temporarily unavailable
error: openssh-server-8.0p1-13.el8.x86_64: install failed
error: openssh-server-8.0p1-10.el8.x86_64: erase skipped
I apparently cannot update the time of sshd_config?
# touch /etc/ssh/sshd_config
touch: setting times of '/etc/ssh/sshd_config': Operation not permitted
Thank you so much for answering! I'd love to use python since I'm way more comfortable with it compared to Bash, but I'm working on a Docker entrypoint script and I think it's best to just use Bash for that.
Anyway, I found a solution. I ended up not using subshells, so that I could use $! and it worked! Thanks again for your python example though ๐
subprocess.run should return a "completed process" object that includes a status code and i think a pid
!d subprocess.run
subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, ...)```
Run the command described by *args*. Wait for command to complete, then return a [`CompletedProcess`](https://docs.python.org/3/library/subprocess.html#subprocess.CompletedProcess "subprocess.CompletedProcess") instance.
The arguments shown above are merely the most common ones, described below in [Frequently Used Arguments](https://docs.python.org/3/library/subprocess.html#frequently-used-arguments) (hence the use of keyword-only notation in the abbreviated signature). The full function signature is largely the same as that of the [`Popen`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen "subprocess.Popen") constructor - most of the arguments to this function are passed through to that interface. (*timeout*, *input*, *check*, and *capture\_output* are not.)
according to documentation only .Popen returns .pid though.
And if it is run with shell=True argument, it returns pid of a shell instead of a process
hm, good to know i guess. at least then don't use shell=True
I'm using nvim with pyright. I have the following code:
class HologramResponse(BaseModel):
success: bool
limit: int
size: int
continues: bool
links: Optional[dict]
lastid: Optional[str]
data: list
def fetch() -> None:
response = requests.get(url, headers=headers)
hologram_response = HologramResponse(**response)
data = hologram_response.data
lastid = hologram_response.lastid
linkid_list = []
month_list = []
year_list = []
sessions_list = []
bytes_list = []
for idx, value in enumerate(data):
linkid_list.append(value.get('linkid'))
month_list.append(value.get('month'))
year_list.append(value.get('year'))
sessions_list.append(value.get('session_count'))
bytes_list.append(value.get('bytes'))
...
So pyright is telling me that I'm have an error (check picture attached) but the code runs perfectly and I do not understand why pyright is telling me that. Can someone explain me? This do not happens on VS code.
Nice, u have typing checker
And it says your code is wrong in terms of typing
In vscode that can happen too with correct installed plugins
But I do not understand why I'm really confused haha, so what should be the right way to write it?
Fix data: list to correct type inside?
What type of objects in list?
list[Any] at the moment
I did that, inside the list is a dictionary. So I updated it to data: list[dict] but it displays the same error
your enumerate is't corect
thats why it says tuple[int, Unknown]
because thats what enumerate returns
just do
for value in data:
...
๐ฎ you are right!! it returns the idx and the value!! that solved my question, thanks a lot!!!!
np
Is there a way to efficiently copy-on-write a file on Linux? Block by block, not copying the whole file on first write like overlayfs does?
Is there a tool to do that? If you're not running a filesystem with that capability already like btrfs or zfs
no COW needs to be implemented on filesystem level
It could be implemented as a virtual filesystem like aufs or overlayfs or unionfs-fuse
but they all seem to copy the whole file if you want to write to it
well that's defintion of COW
you could only copy the parts of the file that you change
thats ROW
not COW and thats why btrfs and ZFS do
most people just refer to it as COW
So there's no way to do this, you have to move the original file onto a new filesystem that has built-in support for this?
well you were right about that you can implement it as virtual filesystem however i cannot find anything that implements ROW
so unless you want to write 1 yourself then there is no other way
I don't think your terminology is right for COW/ROW though
I thought they were only distinctions on which copy is the read/write copy when doing snapshots
e.g. in Linux the COW you get from fork() or mmap() is called COW but it operates at the page level
btrfs, zfs, etc also refer to this as COW
Honestly it does sound like I should make one myself. It shouldn't be too hard with fuse, especially if I don't want to support every possible metadata operation... and if I can rely on sparse files
from my understanding COW copies data on 1st write an then operates on new data like PHP does. While ROW will redirect you to old data if you try to acces part of it that had't been modified after 1st write
I think the only tool that I could use to do this is devicemapper... e.g. turn my file into a /dev/loop0 and use devicemapper's built-in COW to make another device with COW
then I have a device, not a file, but that's close enough
How do i fix CERTIFICATE_VERIFY_FAILED error
most of the solution refer to running Cerificates.command
is there an alternative way to fix this
i installed python using macports
is this a macports issue?
maybe, I used the installer
/Applications/Python 3.10/Install_Certificates.command @main olive
that's where mine is
my cert installer
i installed it via macports
which install python in diff location
/opt/local/bin/python
vs code?
one sec
it just in appliation folder
no idea
application folder
look in there\
Install_Certificates.command
you are looking for that
or wait
do you have pip
yes
one sec
lemme send you my file
the cert installer
my mac is really slow
my python folder
Hey @shy igloo!
It looks like you tried to attach file type(s) that we do not allow (.command). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a, .csv, .json.
Feel free to ask in #community-meta if you think this is a mistake.
@main olive i'll dm it to you
python the bot stopped me
orrrrrr
not
okay i can't dm you
uhhh
one sec
#!/bin/sh
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 << "EOF"
# install_certifi.py
#
# sample script to install or update a set of default Root Certificates
# for the ssl module. Uses the certificates provided by the certifi package:
# https://pypi.org/project/certifi/
import os
import os.path
import ssl
import stat
import subprocess
import sys
STAT_0o775 = ( stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
| stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP
| stat.S_IROTH | stat.S_IXOTH )
def main():
openssl_dir, openssl_cafile = os.path.split(
ssl.get_default_verify_paths().openssl_cafile)
print(" -- pip install --upgrade certifi")
subprocess.check_call([sys.executable,
"-E", "-s", "-m", "pip", "install", "--upgrade", "certifi"])
import certifi
# change working directory to the default SSL directory
os.chdir(openssl_dir)
relpath_to_certifi_cafile = os.path.relpath(certifi.where())
print(" -- removing any existing file or link")
try:
os.remove(openssl_cafile)
except FileNotFoundError:
pass
print(" -- creating symlink to certifi certificate bundle")
os.symlink(relpath_to_certifi_cafile, openssl_cafile)
print(" -- setting permissions")
os.chmod(openssl_cafile, STAT_0o775)
print(" -- update complete")
if __name__ == '__main__':
main()
EOF
copy that into a .command file
and run it
if that fails then switch from python macports to python official installer
bc i think the dir at the beginning references an executable that isn't there
didnt work
i'll make an ticket on macports support
switch the /Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10 for where your python is
@main olive try that first
how?
wait
imma make a python file for you
so you can just python it
# install_certifi.py
#
# sample script to install or update a set of default Root Certificates
# for the ssl module. Uses the certificates provided by the certifi package:
# https://pypi.org/project/certifi/
import os
import os.path
import ssl
import stat
import subprocess
import sys
STAT_0o775 = ( stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
| stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP
| stat.S_IROTH | stat.S_IXOTH )
def main():
openssl_dir, openssl_cafile = os.path.split(
ssl.get_default_verify_paths().openssl_cafile)
print(" -- pip install --upgrade certifi")
subprocess.check_call([sys.executable,
"-E", "-s", "-m", "pip", "install", "--upgrade", "certifi"])
import certifi
# change working directory to the default SSL directory
os.chdir(openssl_dir)
relpath_to_certifi_cafile = os.path.relpath(certifi.where())
print(" -- removing any existing file or link")
try:
os.remove(openssl_cafile)
except FileNotFoundError:
pass
print(" -- creating symlink to certifi certificate bundle")
os.symlink(relpath_to_certifi_cafile, openssl_cafile)
print(" -- setting permissions")
os.chmod(openssl_cafile, STAT_0o775)
print(" -- update complete")
if __name__ == '__main__':
main()
@main olive โฌ๏ธ
copy that into a .py file
and python <file> it
i guess if that doesn't work you can check with macports support
https://andatche.com/articles/2012/02/fixing-ssl-ca-certificates-with-openssl-from-macports/
found a solution
just need to install curl-ca-bundle
ah
Just ran the code
didn't get any errors
thx
which theme
which wallpaper
Hi pythonistas! Can anyone gather why this would give me duplicated output? ```import subprocess
Get a list of files in a directory
output = subprocess.run(("ls","../assets/"),capture_output=True, text=True)
for i in output.stdout.split('\n'):
print(i)
it prints the list of files twice...very bizarre
This is so bizarre I'm wondering about my sanity. I rewrote it using os.scandir and am getting a similar result
can you post full command?
also```(my-env) bb@s76:~/wrk/game-reps/blaster-2d/triangle$ ls ../assets
b2d b2d.ldtk block1object.svg enemy1object.svg enemy2object.svg wand_object.svg
(my-env) bb@s76:~/wrk/game-reps/blaster-2d/triangle$
@dapper musk thank you for your help. I'm going to shut my box down and do something else I feel like I'm in a stephen king movie
I'll ping you on this later if I get an update
ok
@dapper musk Phew...not insane. I was importing a module called triangle while inside of a directory callled triangle
so it was importing itself .. ? ;P
wild but fixing the directory names solved it
oh i thought that importing itself would return an error
well apparently it does't
import main
print("test")
python3 main.py
test
test
well for some reason this works
I'm using ssh to upgrade Python on a Raspberry PI running PI OS (Debian). I used sudo make altinstall and that is running right now. does this actually install the new version or is there an extra step to install it? The tutorial is not very clear
The tutorial says the next step is to delete the old link and then run this sudo ln -s /usr/local/bin/python3.10 python3 but that implies the new Python was installed?
nevermind, I found another tutorial which says that sudo make will install the new Python
Woo it worked! ```pi@raspberrypi:~/Python-3.10.7 $ python3.10 --version
Python 3.10.7
this probably isn't what happened. show your file layout and describe how you were running it
you might want to try pyenv!
or at least python-build
I already showed the file structure and also Hodd Toward reproduced it. Also already using miniconda. Also changing the folder name one character fixed it... Pretty solid evidence that is what occurred
i apparently missed the context then
usually these issues revolve around having the wrong cwd and not understanding how python searches for modules/packages
@formal schooner its good point you make. from sys.path it shows the cwd being the first place it looks for packages. The unexpected result that occurred for me is that the cwd had the same name as a different module I was importing for a location farther down on the list. very weird
yeah, python loads the first result it finds in sys.path
so if you have a globally installed module foo, as well as ./foo.py, and ./foo/__init__.py, import foo will find and load ./foo/__init__.py and completely ignore the others
other issues arise when people run python foo/a.py and don't understand why things behave differently different from python ./main.py or python -m foo.a
What is the advantage over installing Python versions how I did?
don't need to fuss so much with figuring out dependencies and compiler flags, or with downloading and extracting tarballs
usually people recommend first using python venv, or python-venv which is the same within python ecosystem
this solution is preferable because it ensures not polluting your OS global python installed packages
this solution is still horrible for any deployment though
like pyenv (which is advanced version of venv, but i dislike it for long deps resolving)
the real solution is to use docker containers ๐
they ensure all dependencies installed for applciation run (python packages / installed binaries to OS, whatever) and it would be still not polluting global OS system
each application can be having their own crap installed, without conflicting with others
and their old version would be easily wiped without a trace from system, when you deploy a newer version of them
Yeah I had to install a few things manually to get it to work. So pyenv installs Python to a specific folder. Do I still use venvs? Just use the Python versions I installed to make a venv?
Yeah I'll probably use docker if I get an actual web server deployed on this thing
venvs are still useful but less necessary, since the installed python is entirely unaware of packages installed by other pythons
So I would use pyenv on a per-project basis? Like make a new install for each project?
a favourite newbie deployment package is to use docker-compose usually. Really comfortable experience with boilerplating docker crap in yaml ๐
minimal deployment final command: docker-compose up -d to deploy it, docker-compose logs to get its logs, docker-compose up to run without running in background
and no need to set any other multiple parameters like in regular docker
yeah i use docker compose even when i only have 1 image
podman supports it now too i think
i'm still using docker because pip caching with buildkit is awesome and i've read mixed reviews of podman support for buildkit
i am tempted to migrate my dev environment to kubernetes one day. There are already made tools for that like tilt, dev spaces
but for now i am sufficiently satisfied with docker-compose too
I'm running uvicorn through ssh, but the server closes when I end the ssh session. Any easy workaround for this?
Just a google lead would help
google: ubuntu 20.04 create systemd service -> https://www.how2shout.com/linux/how-to-create-a-systemd-service-unit-file-in-linux/
(at least if u use ubuntu or debian based OS images for your server, for other linux OS types solution is a bit different how to schedule daemon runs in background)
Debian/Ubuntu is easiest to manage though
technically there can be even easier solution than this, but they are more ugly and less intended way to have it. This way ensures to set auto restart of a service on reboot or when ended in errors, amd seeing its logs
Thank you! I'm running ubuntu server
Is it normal to still need to use "python3.10" after setting 3.10 as local? ```bash
pi@raspberrypi:~/servers/test_fastapi $ echo "import sys;print(sys.version)" > main.py
pi@raspberrypi:~/servers/test_fastapi $ pyenv versions
system
- 3.10.7 (set by PYENV_VERSION environment variable)
pi@raspberrypi:~/servers/test_fastapi $ python main.py
3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110]
pi@raspberrypi:~/servers/test_fastapi $
pi@raspberrypi:~/servers/test_fastapi $ python3.10 main.py
3.10.7 (main, Sep 14 2022, 02:12:25) [GCC 10.2.1 20210110]
``` using the full command works
no, did you forget to add the shell integration to your shell and restart?
I'll restart shell. I added all of the lines to .bashrc
eval "$(pyenv init -)" ; eval "$(pyenv init --paths)" or something like that
yeah you need to open a new shell for bashrc to take effect
Ahh now it works! I had to restart the shell after installing the new version, thank you
shouldnt be necessary every time, only after first installing pyenv
Well I thought I had already restarted the shell after installing pyenv
weird. you definitely shouldn't need to restart every time
This is in my .bashrc ```bash
export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
you will want to guard the eval behind command -v pyenv too but yeah seems fine
It seems to working normally now, maybe I had previously entered "pyenv shell ..."
oh that's also possible
:incoming_envelope: :ok_hand: applied mute to @supple hare until <t:1663175157:f> (10 minutes) (reason: burst rule: sent 8 messages in 10s).
The <@&831776746206265384> have been alerted for review.
@supple hare spamming to try and get voice verification is a quick way to have another few weeks of voice mute. Please don't do that.
I'm trying to set up an alias in Powershell to quickly startup a VM in VBox
Command to start a VM in VBox: vboxmanage startvm Fedora where Fedora is the VM name I wanna start
I tried Set-Alias -Name startfedora -Value vboxmanage startvm Fedora but it couldn't accept 'startvm' for some odd reason
Set-Alias : A positional parameter cannot be found that accepts argument 'startvm'.
At line:1 char:1
+ Set-Alias -Name startvbox -Value vboxmanage startvm Fedora
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Set-Alias], ParameterBindingException
+ FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.SetAliasCommand
sounds like you need some quote marks, there, Son
have you tried Set-Alias -Name startvbox -Value "vboxmanage startvm Fedora" or similar?
one sec
It saved it but the shell doesn't recognize the value
gimme a min
It does work if I run it manually
This is what I got after I set startfedora alias
startfedora
startfedora : The term 'vboxmanage startvm Fedora' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of
the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:1
+ startfedora
+ ~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (vboxmanage startvm Fedora:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
But running vboxmanage startvm Fedora does work manually.
might want a function instead of an alias
In bash, I never used aliases, and used functions isntead, for more or less this reason. Be funny if the same problem, with the same solution, is present in powershell
dunno
hmm
try PS> help about_functions
It worked!
function Start-Fedora { vboxmanage startvm Fedora }
oh a function worked. That doesn't surprise me.
function Stop-Fedora { vboxmanage controlvm Fedora poweroff }
use your power only for good
๐
lookit you, following the naming conventions 'n' stuff ๐คฃ
looks around
this is #unix, right?
I mean I know pwsh runs on Unix, but ...
Hence why I asked here innit
surprised there isn't a #windows -- seriously there should be. Far more people use it, and it has its ... uh ... unique challenges
I'm surprised not alot are asking WSL related questions
there used to be, i think it was removed because it got a lot of spam from people who didn't read the channel description
Windows isn't only WSL
FYI for a linux guy
What do you mean?
It's literally Windows Subsystem for Linux
WSL has some weird issues vs Linux
Hello I was wondering if anyone can help me. Im trying to install python3-pip package on ubuntu and it does not work
anyone knows whats the problem?
no ubuntu
and do you know why is it doing?
it is corrupted
Is there a better way to use entry-points with selinux rather than having to install as root?
I'm guessing not but figured it was worth seeing what other people had tried
is there an error log of some kind?
i don't know much about selinux, but there has to be some way to allow the python entry point to work without installing as root
The thing with selinux is part of its security measures is not allowing users to execute things that aren't in a root executable directory... Which completely defeats /home/.local/bin/
i'm pretty sure that selinux can be configured to allow users to execute binaries in other locations. but i'm not sure how
it looks like you need to create a new context for the executables in your home dir, e.g. semanage fcontext -a -t jahrules_user_bin_t /home/jahrules/.local/bin, and then set an allow rule for that
Hi guys, I recently installed https://dbgbench.github.io/ tool. And I was trying to investigate some of their files for say find.07b941b1.report.txt in their repo to find few information like program, bugid, bug type, fault location, fault condition like this. Any suggestion what command I should use to find those information. I was using the run.sh in the docker dir but its not working for me. Any help regarding this. Thank you
I was trying to find few information like
program, bugid, bug type, fault location, fault conditionlike this.
where os this information stored? you can probably grep the file to get what is needed
Thanks. I able to get the information I need but it looks ugly to read ๐ trying to understand meaning of each information from it.
are help messages usually sent to stderr? I was looking at https://github.com/google/styleguide/blob/gh-pages/cpplint/cpplint.py#L6129
if you will write as `sys.stderr.write("text"), it will get sent to stderr ๐
if you will write through print, it will go to stdout (though u can specify in print as destination being to stderr too)
if you will use sys.stdout.write, it will go to stdout
if you will use raise Exception("text"), it will make tracelog to sderr
and no. help messages aren't usually sent to stderr. But if user inserts wrong action, he receives interface help sent to stderr yeah
I'm trying to fiddle a log file to yank certain and text and write with yanked text. I understand there's a register in vim but I can't seem to figure out how to yank and assign to different registers. Is there something in docs for this? Hard to find.
"ay would yank to register a
"ap would paste from register a
ah
thank you!
I've been doing "+y to save to system register and have that mapped to a vnoremap
I didn't know how it worked
:h registers should tell you more
thank you!
ok so its not what I was hoping for. I'm struggling to (first) yank the text in the file and (second) yank another text in the file and write the second yank to the first yanked text. Got any idea?
I can do this with a macro of searching and visually selecting, but I'm having trouble with :w to write the filename from the first yank
like, append to the register?
swap?
you mean a swap file?
no swapping register contents
oh no. I wanna save the contents
One min
<xml>
...
</xml>
filename: blabla.log
I wanna copy all content between and including <xml> tags and save them to the filename blabla.log
This is in one huge log file
So I wanna save the many log files that its saving into multiple xml log files
Hope that make sense
I can do macro and visually copy between the xml tags and the filename. I'm just having trouble saving the content to the filenames mentioned in the same huge log file
Otherwise, I might try grepping but not sure how to grep multiple lines for the xml tags
I gave up and just did the whole in Python.
My lack of bash/vim scripting has failed me
yeah, I don't think that's something you should be doing in Vimscript anyway
xml is the classic example of something that you cannot and should not try to parse using regex and other "context-free" string processing. you really do need a proper parser, unless you can make specific assumptions about the structure of the document.
so it's pretty easy to write a sed script to copy text between two lines, but once you have to start extracting information from the xml itself, you're struggling
@weak sorrel is this what you wanted?
#!/bin/sh
set -eu
input='test.xml'
output="$( sed -En '/filename: / s/filename: (.*)/\1/p' "$input" )"
sed -En '/<xml>/,/<\/xml>$/ p' "$input" > "$output"
oooh. I did try awk but couldn't handle the multiline between the xml tags thing. I assume , is nothing?
what do you mean?
i also was just writing up an awk solution that does it in a single pass over the data
what is , in /^<xml>/,/<\/xml>$/ p?
that's a range of patterns
oh
oh the ^ probably doesn't need to be there
it's just pat1,pat2 ... meaning "apply ... to all lines starting at pat1 and ending at pat2, inclusive"
you see it with numbers too, like sed -n '5,$ p' is equivalent to tail -n 5
what about it?
Couldn't we do multiple patterns with |? Is it better with , instead?
that's completely different
I've not practiced with ,
oh
, isn't part of the regex pattern syntax. it's part of sed syntax
in regex syntax , is just a literal , character
well thats why I didn't recognize it
I guess I need to brush up on my sed and awk
I might need them in the near future
very useful tools
The Grymoire - Tutorial on the SED stream editor.
The Grymoire - Tutorial on the AWK program language.
nice little awk solution
#!/usr/bin/env -S awk -f
BEGIN {
run = 0
n = 0
output = ""
}
/<xml>/ {
run = 1
}
run > 0 {
lines[n] = $0
++n
}
/<\/xml>/ {
run = 0
}
/filename:/ {
output=$2
}
END {
if (output == "") {
for (i = 0; i < n; ++i) {
print lines[i]
}
} else {
for (i = 0; i < n; ++i) {
print lines[i] > output
}
}
}
it's only a single pass over the file, but it does require holding the entire output in memory until you find the filename at the bottom
the awk solution isn't much different from the equivalent python solution, just different syntax and the for line in sys.stdin is implied, not written out
personally i think awk is much easier to learn, and its functionality is more or less a complete superset of sed. i'd start there first
I was told the same about awk too
fun story:
once when i was an intern, the team had received a data file from some tech people, but nobody on our team knew how to open the file (they were all excel users) which had the extension .dat, and the tech people weren't very responsive.
i had just recently started learning about unix tools, so i decided to try looking at the file with head and less, at least to visually inspect if it was "binary" or "text". it turns out that it was just pipe-delimited text data like col1 | col2 | col3. i ended up writing an awk script to clean it up, then i loaded it into R for further processing, and finally exported to an Excel book. i was treated like a hero for that!
What is a non-cursed way to check if something is a valid PCRE in python? (I'm guaranteed to be on Debian and have some normal packages).
I was able to do it by calling Popen(["grep", "-P", my_thing], ...) and seeing what error code it returns. But like, on a scale from 1 to cursed this is pretty cursed. Also not very fast.
I probably don't need it to be scientifically proven or whatever.
in hindsight, fuck you to whoever gave that dataset to my team without helping them open it and/or converting it to xlsx for them to use. but it was a great opportunity to apply all the stuff i had just learned, and to feel the raw sexy power of "the unix philosophy"!
other than binding to libpcre i don't think there's a better way. parsing the regex itself seems like a deep dark pit of wasted time and frustration
hmm makes me think about files from libreoffice with odt or whatever its called
Ah yes, ordinary differential tequations
i have no idea how the odt format is laid out, but it's probably not parseable as plain text. i know that docx, xlsx, etc. are some kind of weird XML/ZIP hybrid
Yeah I was just wondering if there's anything in the cpython stdlib that already uses pcre.
nope i am pretty sure python uses its own separate regex engine and cpython knows nothing about libpcre
Like how sqlite uses btrees
for the most part i think python regex is a subset of pcre regex
not sure if it's a strict subset though
Yeah they're different
Maybe grep isn't that bad... And the program may be using grep in some cursed way already
oh dear are you doing some kind of input validation for a different program
i wonder if there's like an ABNF spec for PCRE somewhere that you can implement with Lark ๐
(subprocess doesn't look so bad by comparison)
Oh hold up @formal schooner, I can just call into libpcre2 with ctypes can't I
(""just"")
pcre2_code *pcre2_compile(PCRE2_SPTR pattern, PCRE2_SIZE length, uint32_t options, int *errorcode, PCRE2_SIZE *erroroffset, pcre2_compile_context *ccontext);
"just" indeed
how's that less cursed than calling out to grep? ๐ฟ
Should be faster innit?
how fast do you need it to be?
I need to validate regexes from an API request, which might be hundreds, hypothetically. So maybe under 1ms
Ohh nice. The grep version takes about 1.5ms (as expected) while the ctypes version is only about 15us.
Ye
Wdym how? Is there more than one way with ctypes?
I'm loading the libpcre2 library and then calling the pcre2_compile_32 function. If it returns a pattern, I'm just freeing it. Otherwise, I raise a Python exception with the error message
Also the error position is helpfully included unlike with grep which is just like: "you have an error in your regexp! where exactly? fuck you, that's where!"
oh i mean figuring out all the right inputs and such
i'm a little surprised that there isn't already a maintained libpcre binding library, but i guess it's usually not needed in python since python regex is close enough
What is the default root logger?
For example, logging.info() logs a message to root logger, but I am not seeing it anywhere. I've checked syslog, messages, debug, etc.
So, if I pipe it to a file... Hm.
Wait a minute.
2>&1 suppresses stdout, doesn't it?
no
this is probably the issue you're facing
The real issue I'm facing is that if the default level is warning...why am I even logging it?
What I could do is set the level to be set at INFO, IF debug is True.
Because when it works as expected, I don't really need to see it. It's a CSV line that it puts out if it matches another file.
you could set it to DEBUG if debug else INFO
I was just looking at this, thinking, "If only there were some place it was documented..."
At SetLevel() where it says it only logs at WARNING. ๐
Okay, thanks. Now that I know what's up, I'll finish doing this step.
by default, yeah
logging.Logger.setLevel("DEBUG")```
This is throwing an error.
It says the Level is unfulfilled.
try
logging.getLogger().setLevel(logging.DEBUG)
Okay, that works and PyCharm is pulling information for each part of it. logging.info(str(line)) So, when this is called later, if debug == true, it'll automatically get stdout?
I convert it to a Boolean. I call it from a configuration file, so I have to convert it in the code.
I learned that the hard way.
So, to my code up there, I have to add debug = True.
!pypi pcre
!pypi python-pcre
and v0.7
I mean, maybe it still works
Not sure I get you
I just read the docs of libpcre2
i guess its just a matter of reading the docs, but figuring out all of whatever this is
pcre2_code *pcre2_compile(PCRE2_SPTR pattern, PCRE2_SIZE length, uint32_t options, int *errorcode, PCRE2_SIZE *erroroffset, pcre2_compile_context *ccontext);
and getting it to "not segfault"
thankfully I know how to use valgrind ๐
i definitely do not. i should learn maybe
wait let me get it in a moment
you just run valgrind your_program that's it
and it does magic segfault diagnostics
oh cool
like, valgrind python ass.py
i guess the assumption is that python itself shouldn't have memory leaks, so anything that does leak would come from calling c functions?
yeah
well it's not about memory leaks
basically, when it segfaults, valgrind shows you the stack trace
like, in which function it died
oh wow
Well it's something like
error_code = c_int(0)
error_offset = c_size_t(0)
booowoa = b"".join(ord(s).to_bytes(4, "little") for s in string)
adr = pcre2_compile_32(c_char_p(booowoa), c_size_t(len(string)), c_uint32(0), pointer(error_code), pointer(error_offset), c_void_p(None))
if adr is None:
# inspect error_code and error_offset
else:
pcre2_free_32(adr)
I have done a bit of C so nothing complicated
yeah cool. i guess it's just a matter of looking at the docs / header file and figuring out what all the typedefs are
That's a reason not to like ctypes, not a reason not to like C. The whole point of a typedef is to simplify things for the end user by aliasing type names. You're not supposed to need to unroll them, the compiler is supposed to do it for you
you're facing that because ctypes deals with the library's ABI, rather than with its API, and typedefs are part of the API.
it prints them unescaped, which is why you get that behaviour
you can, for example, print the whitespace and newlines by passing -A
I wrote a thread.
It works fine.
Also, i have try & except blocks for keyboard interruptions.
But still i see an error
The threading code:
The error:
@velvet estuary #async-and-concurrency is a better place for this question
Does anyone know if its possible to use multiple cronjobs with different timezones?
Firstly yes, it should be possible when using docker
Secondly... i think u should use celery beat as a more customizable solution
im not using docker
celery beat looks promising
good time to learn anyway. it augments any solution
Docker Deep Dive is good book to get started
makes easier to use celery beat too
u just have one docker compose, where one container is celery beat, another celery worker to process tasks, and preferably celery flower as monitoring system. And redis key-value as meesage broker (has queued tasks inside before they are processed)
So at least three processes, which would be looking much better in docker
#!/bin/bash
total=0
find /root/Desktop/code -type f -name "*.[ch]" | while IFS= read -r file; do
count=$(grep -c ^ < "$file")
echo "$file has $count lines"
total=$count
done
echo TOTAL LOC: $total
Hi guys, need help with this script. So I am able to print lines of code but the problem is it's not adding all the c and .h files lines at the end. I was trying to get the total LOC from this script. Anyone would like to help please:) Thank you.
that's because you're doing total=$count
I tried with $total=$(($total+$count)) not working
does anyone know a good program for benchmarking in linux?
do total=$(( total + count ))
benchmarking what?
cpu and gpu
Ok, I'll give it a try and see if it works or not:) Thank you again
I only know of stress-ng, but that's CPU-only I think
thanks
because its cold for me and im on laptop and i need warm and i dont like gas prices
๐
heyo, i have a discord bot that im hosting on amazon aws. this question isnt related to discord bots though.
i want to make my python bot script run 24/7, and im using tmux for this. the problem is, when i run my script, the terminal becomes non interactive and im unable to detach from it to make it run 24/7. how should i solve this?
The correct solution : https://unixcop.com/how-to-create-a-systemd-service-in-linux/
To get unstuck without closing your bot, you can Ctrl+z
sucukmitei@sucukmitei-S15447:~$ sudo mount -t ntfs /dev/sda1 /mnt/D
The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
Falling back to read-only mount because the NTFS partition is in an
unsafe state. Please resume and shutdown Windows fully (no hibernation
or fast restarting.)
Could not mount read-write, trying read-only
can someone tell me why i cannot mount /dev/sda1? i need to mount it so it can be written
As it says in the error, there is a problem with the file system. If you can mount it to a Windows device and run chkdsk it may be able to find and repair the problem
This is a drawback of NTFS in my experience, it will happen occasionally
Better to use a native Linux filesystem if you can and share to windows via smb or nfs
It saidbthere are no problems
@ember quiver what should i do
You might be out of luck, but you can Google around
If it mounts fine in Windows I would share it over network via SMB probably
How do i Share ober Network because both Windows and linux are in the same machine
Ah, not an option if you're dual booting.
yes i am using dual boot
i found this on stack overflow i will try
okay it workd
thhanks dowcet
Hey.
Here's an an example service for systemd (name it mysrv.service, or anything but it has to have .service at the end):
[Unit]
Description=A service
After=network.target
[Service]
Restart=always
User=user
WorkingDirectory=/home/user/bin/user/
ExecStart="/home/user/bin/user/main.bin
[Install]
WantedBy=multi-user.target
And here's a shell script to place it to the right place and enable it (name it installsrv.sh):
# !/bin/bash
# $1 = service file name (without extension)
ln $1.service /etc/systemd/system # Create a link instead of copying here
systemctl enable $1
Example usage: ./installsrv.sh mysrv (If you used a different name for the file above, use that instead of mysrv)
After this you'll have to start the service: systemctl start mysrv (or restart to restart, stop to stop)
anyone here has experience with vim?
a little
I getting this error while installing packages
in arch linux
How can I solver this?
Can your system connect to the internet?
Did you follow the installation steps to set up an internet connection?
how do i check it
ig no
how do i RESOLVE THIS
I was alluding to that with this message
yup
everything working fine
now i was just going to setup custom kernel
for that I was installing some packages and now I getting just errors
I don't have experience with setting up a custom kernel.
I would suggest you double check the configuration of your network manager
sure
And make sure your VM is set up to provide internet access
Make sure your network manager service is actually running
Could also be a firewall misconfiguration
This page may be helpful https://wiki.archlinux.org/title/Network_configuration#Check_the_connection
Anyone have tips for cleaner CLI formatting? Or is that something where as I write more scripts, I just do the same thing over and over again?
I've started putting like * [I] Information Goes Here for example.
do you mean, you want to know how to format your tool's output?
That made me giggle at first because I am immature, but yes.
if the output wants to be a table, use https://tabulate.readthedocs.io/en/latest/ -- super-easy to use; output looks good
How would you format doing like:
Now it's doing this
And also this
Oops! Have an error here!
But we can work around it```
well that's not particularly tabular, so I'd format that just as you've shown
what problem are you trying to solve by improving the formatting?
Purely aesthetics.
gotcha
When I run a lot of scripts, they just feel cleaner than that.
so copy their formatting ๐
Maybe it's just that they picked a standard of output and stayed with it.
I didn't know if there was a document somewhere with standards or if it was made up.
you could use the logging library -- that won't make anything prettier, but it'll add timestamps and INFO and stuff
Logging is a handy one to use. I like it.
>>> logging.info("See?")
2022-09-28T14:37:10+0000: [000000000018243] See?
>>> ```
I'll take this all in.
Is there a smart way to setup sudo so it doesn't prompt me for passwords? Maybe a yes/no popup instead?
I don't want to enter my password when I type 'sudo' in a shell, but I don't want program to be able to use sudo without my noticing either
In the sudo config you can setup using sudo with nopass
And you can configure it for only specific commands/users/groups
It's quite granular if you need it to be
yeah but does that prevent some random program from running "sudo"?
Well no; but you can essentially use the config to permit certain sudo operations that aren't particularly dangerous for a program to run
how about raising docker container and having free reign from the name of root user for all programms
it will all dissapear with container deletion anyway. Your host OS will remain safe.
Yeah that's a problem, I don't want any process running under my user account to be able to run commands
But when you run sudo the first time; for the next 15 minutes (default) any subsequent command also doesn't need a password
yes but that doesn't include other terminals and it does include other processes in my terminal beside my shell
that's not what I want
Convenience and security rarely go hand in hand.
I don't know if I believe that
I could get a Yubikey and have it enter the password when I press it. I sort of just want to skip the password step
it provides no security in this setup
why not open another window, and type "sudo -s" into it, and just use that when you feel the need ... the need for root?
Do you actually need root that often that this matters?
Or is it an xy problem?
Just setting up my machine, just updated. This is one of the inefficiencies I'm trying to remove
You know back in my day, sudo didn't exist and we liked it
Everyone just used root like real sysadmins
su -c was always a thing
Anyway, I fail to see how an occasional password entry is an inefficiency for anyone with semi-competent typing skills
It's like 15 extra chars max
I found polkit which sounds promising. It knows how to ask my graphical session for a password. I don't know how to replace the password prompt with an ok/cancel though
I'm pretty sure password prompts aren't meant to be bypassed
We had to change the UID ourselves, by patching the kernel
xD
Stage 1 Gentoo bootstrap install!
You try telling that to the younger generation ...
I setup full-disk encryption and enabled auto-login
I don't put passwords everywhere for fun, I have a threat-model in mind, that's how security works
I might just set NOPASSWD and forget about this, it's weird that there is no option though. Windows has a security prompt without a password, I thought Linux would be capable of it
As if Windows were a good role model for adequate security?
that's no argument
in theory you can just write setuid program that will prompt for yes/no and use env to run whatever program you want. it will be secure enough because any malitious program acting would need to be targeted in order to use it and well propabilty of this being the case is basicly 0
you can also try appending
auth required pam_securetty.so
...
account required pam_securetty.so
...
session required pam_securetty.so
and try to make setuid wrapper that will just append tty to /etc/securetty and then set NOPASSWD on sudo
this will refuse authentication unless tty on which sudo is called is in this file
Who can help me with a Linux question
maybe me, if you post text instead of a photo ๐
Okay give me a few minutes
surely you're not typing it by hand?
No Iโm taking a screenshot
uh
I don't want a screenshot; I want text
so that I can run it myself, without having to type it
How would I do that?
use the clipboard to copy the text from the console, or text editor, or wherever it is; then paste it into discord
Okay let me see if it lets me
#!/bin/bash
read -p "Enter the city name" city
for temp in Monday Tuesday Wednesday Thursday Friday Saturday Sunday
do
read -p "Enter the temperature of $temp" $temp
done
let Total = $Monday + $Tuesday+ $Wednesday+ $Thursday+ $Friday+ $Saturday +$Sunday
let Average = $Total/7
echo $Average
Like that?
yep, tx
Okay Iโm just trying to get an average of all 7 days
It keeps saying the bottom 2 lines are wrong
here's what I see
(echo 10; echo 20; echo 30; echo 40; echo 50; echo 60; echo 70) | bash unix.sh
unix.sh: line 7: let: =: syntax error: operand expected (error token is "=")
unix.sh: line 8: let: =: syntax error: operand expected (error token is "=")
is that what you're seeing?
i didn't even know bash had a let builtin
it looks like the syntax is a bit fussy and apparently requires some quoting https://wiki.bash-hackers.org/commands/builtin/let
try Total=$(( $Monday + $Tuesday + ... ))$ instead (filling in the ... with the rest of the variables)
the $(( )) is "arithmetic expansion"
no i am saying you should consider using something else instead of let, because the syntax will be easier. the doc i linked has one option, and i just proposed another
and I am saying you should consider using something else instead of bash ๐
like, I dunno, I hear python is pretty nice
also valid. learning shell scripting is a useful skill though
this arguably is not a great use case for it
the power of shell scripts usually lies in the ability to seamlessly interoperate with files and text streams, and generally low syntactic overhead for managing background and child processes
your homework is to write a shell script, but you don't understand how shell scripting works?
admittedly there isn't a whole lot to understand. Bash is a computer program that reads lines of input, and performs certain operations based on the commands provided in the input
He hasnโt taught us that
the input is a programming language
We only know about nano and pico
bash scripts are more or less the same as typing each line into the shell one after the other
don't think about it too hard
So if I do (( Monday + all the days)) that should fix the error?
the error arises because the let command expects very specific syntax, and you provided incorrect syntax
So what are my other options if I donโt use let?
if you want to use let, i think you just need to put quotes around the whole $Total = ... part
it looks like it expects each "expression" to be a single "word", which is shell script terminology for what we might consider a "string" in another programming language
But those should do the same thing or does let have a different purpose?
(( .. )) and let are similar and i think maybe equivalent. $(( .. )) is a bit more restricted and thus simpler
Got it let me try the other way Iโll let you know if it works
i suggest reading the page i posted earlier instead of guessing!
Yeah it still doesnโt work
Even if I remove let
Your post is a bit to advanced for me right now
Can anyone else help me a bit more?
did you try putting the whole expression in quotes?
let "Total = $Monday + ..."
did your instructor actually tell you to use let?
Iโm putting โTotalโ = $ โMonday + Tuesday โฆ sundayโ)
Or is that not how you put it
no, the whole expression needs to be a single "word", so it all needs to be in the same pair of quotes
i demonstrated above
So it would be โtotal = $monday+ $ tuesday โฆ $ Sunday โ
yes, try that
And same thing for average?
(keep in mind that Monday and monday are not the same thing)
Yeah the caps matters
yes
basically let has its own mini programming language that you might say is "embedded" in the Bash language
Okay so no ( ) for either of them?
the (( was an alternative option, but it's probably best to keep working with the command you are already working with
Yeah I looked into let us really confusing
(( Total = $x + $y )) is the same as let "Total = $x + $y" -- note that the quotes are not necessary in the first version
and Total=$(( $x + $y )) is yet another equivalent way to do it (note there must not be spaces around = in this version)
Wow so thereโs 3 different ways
if you aren't required to use it, i suggest that you avoid using it
let as far as i know was considered a design mistake and was quickly replaced with ((
this was in the Korn shell during the 80s, and Bash i guess adopted both let and ((
Okay I just tried this
#!/bin/bash
read -p "Enter the city name" city
for temp in Monday Tuesday Wednesday Thursday Friday Saturday Sunday
do
read -p "Enter the temperature of $temp" $temp
done
"Total = $Monday + $Tuesday + $Wednesday + $Thursday + $Friday + $Saturday + $Sunday"
"Average = $Total/7"
echo $Average
That gives me a different error now ๐
in general, you need to show the actual error, don't make people guess
!code also, read below carefully for instructions on posting code with nice code formatting:
Here's how to format Python code on Discord:
```py
print('Hello world!')
```
These are backticks, not quotes. Check this out if you can't find the backtick key.
however in this case the error is obvious: you omitted the actual let command
Wdym?
i think you are relying a bit too much on directly copying the text in my messages, rather than pausing to think through what i am telling you
you didn't actually write let anywhere...
Oh if you do โ โ you have to use let?
you're thinking about this completely backwards. the " stuff was specifically describing how to use let
I thought you said let is same as (( ))
yes, but you didn't actually write let!
you wrote "Total = ..." but you needed to write let "Total = ..."
Oh I see
just writing "Total = ..." as the first word in the command line will cause the shell to try to run a command called "Total = ..." which probably doesn't exist on your system (or anyone's)
Got it that makes sense
@main olive i also just saw your question in the help channel
is that where you got the idea to use let?
No I was watching a YouTube video for loops
And they recommended using let
My professor doesnโt really teach he just makes us read out of a book
i don't understand this obsession with watching YouTube videos for basic information that is probably in your book
it seems to be endemic among young people
the person making the video is pretty much just reading out of the book most of the time
The book didnโt really explain the topic
Like I still have no idea what -p is used for
But I just know you need it
what book are you using?
The Linux command line by no starch press
it might be a good time to start practicing reading the actual manuals
Bash has an extensive manual documenting all of these commands
With man?
the man page for Bash in particular is hard to read and search through because it's all one big page and man doesn't have good support for sections, jumping to specific items, etc.
Bash Builtins (Bash Reference Manual)
the read builtin command is documented on that page
and the -p option thereunder
this page is also a little annoying because it doesn't have html "anchors" that allow me to link directly to the command, but it's easier to read than the man page i think
That makes sense hmm also can you explain to me what | I know itโs the pipe command I still donโt know what it does.
does the book not explain it?
It does but I donโt get it
what does the book say about it?
Iโve been really struggling with the point in piping 2 commands
It just gives examples of like ls | filename
a | b takes the output from command a and uses it as the input for command b
Can you give me an example with a command?
the name "pipe" is meant to evoke the process of connecting output to input with a pipe, like connecting a faucet to a bucket or something like that
no, you don't pipe things to a file
Hmm so when would you use it?
it's for connecting small programs in a "chain" or "pipeline" in order to perform more sophisticated processing tasks
But couldnโt I just do them separately or no?
Is the whole point itโs more convenient?
https://youtu.be/bKzonnwoR2I it might be elucidating to watch one of the actual creators of the pipe syntax talk about their origin!
Just what is a pipeline in the computer science sense? We asked Computer Science guru Professor Brian Kernighan
Why Asimov's Laws of Robotics Don't Work: https://youtu.be/7PKx3kS7f4A
Brian Kerninghan on Bell Labs: https://youtu.be/QFK6RG47bww
Don Knuth on Email: https://youtu.be/QS8qwMna8_o
Computer That Changed Everything: https://youtu.be/6...
Okay Iโll check it out
you could save the output of a to a variable and use that as input to b, yes. there are some advantages to pipes however, including the ability to process infinite streams of text, or extremely large streams that you don't want to store all at once
nowadays computers have a much larger amount of memory than they used to, so it's less important to avoid storing large chunks of text than it used to be
however the general paradigm remains extremely useful
https://youtu.be/tc4ROCJYbm0 here's another gem of a video with mr. kernighan talking about this stuff, back when it was all new
Watch new AT&T Archive films every Monday, Wednesday and Friday at http://techchannel.att.com/archives
In the late 1960s, Bell Laboratories computer scientists Dennis Ritchie and Ken Thompson started work on a project that was inspired by an operating system called Multics, a joint project of MIT, GE, and Bell Labs. The host and narrator of t...
Who is mr kernighan?
the person in the 1st video, and one of the original unix developers
Oh wow is he still alive?
skip to 6:00 in the 2nd video for the demo of pipes
I didnโt know there was videos of the ones who created this stuff thatโs so cool
yep most of the "OGs" are still alive, the oldest are in their 80s i think
Wow thatโs insane also another quick question when doing if elif and else statements does indentations matter or no?
Cuz I know In python it does
not in bash. however it's best to follow standard indentation practices, otherwise your code will be difficult to read
good!
Wait @formal schooner
I do need a bit more help if you got time
#!/bin/bash
read -p "Enter the city name" city
for temp in Monday Tuesday Wednesday Thursday Friday Saturday Sunday
do
read -p "Enter the temperature of $temp" $temp
done
((Total = $Monday + $Tuesday + $Wednesday + $Thursday + $Friday + $Saturday + $Sunday))
((Average = $Total/7))
echo $city $Total $Average
if [$Average -ge 80 ]; then
echo "Summer"
elif [$Average -lt 80]$$[$Average -gt 60]; then
echo "Winter"
else [$Average -le 60 ]$$[$Average -gt 50 ];
fi
Iโm doing if else statements
I keep getting this error
./weather.sh: line 10: [96: command not found
./weather.sh: line 12: [96: command not found
./weather.sh: line 14: [96: command not found
you need a space after [
also use [[ instead of [, the syntax is more flexible
see "[[" in https://www.gnu.org/software/bash/manual/html_node/Conditional-Constructs.html and compare to "test" in https://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.html
Conditional Constructs (Bash Reference Manual)
Bourne Shell Builtins (Bash Reference Manual)
Okay that helps let me see if that works
Wait @formal schooner || is or right?
only inside of [[. outside, it has a different meaning in shell scripts
you can format text with code syntax by surrounding the text with "backtick" characters: "`" these are on the same key as "~" on american keyboards, not sure about european
This user got banned a couple hours back
ah
Hey everyone! Iโm new here on the platform! Nice to meet you all
hi
Does anyone know how i coud strip a URL to extract part of it
Example strings
#https://gitlab.com/stringa/stringb/racing/service
#https://gitlab.com/stringa/stringb/innovation/replayer
If I wanted to take out racing or innovation
From URLs like that?
And return them as an environment variable to the parent script
Try urllib.parse.urlsplit (its SplitResult object's path attribute is what comes after gitlab.com
From there you can split on /
ah sorry i need to use Bash for this @untold socket
echo url | awk -F โ/โ โ{print $6}โ should work however i cannot test it now cause i am on mobile
@modern kestrel in addition to urllib, i really like the Hyperlink library
!pypi hyperlink
oh, bash
not any good way to proceed there other than regex, or shelling out to a python or perl or awk script
it's best to avoid doing too much work in shell language imo. it's best used as glue between other programs
in this particular case you can do it with regex, so sed awk perl python whatever
but none of those are bash, they are all external programs
hmmm, got
awk: 1: unexpected character 0xe2
awk: line 2: missing } near end of file
if you have a particular constraint on your system, eg python isn't available, then state it
URL=https://gitlab.com/stringa/stringb/racing/service
echo ${URL} | awk -F โ/โ โ{print $6}โ
not a constraint per se i can use python to be fair
just curious trying to upskill my bash
use <<< in bash, a "here string", instead of echo |
fair and respectable, but calling out to awk isn't going to help with that either!
my bash honestly feels shocking its just gibberish to me at times
you can probably hack this together using several ${name%pattern} expansions
frankly bash is a nasty messy language to work with. it's only moderately better than posix sh
but its so widely used
zsh is a huge improvement imo, but at that point you're getting dangerously close to "why not write it in python"?
i just know i need to learn it at some point somehow everyone in industry is competent with it
yeah it's absolutely worth learning, but doing things like string processing in "pure bash" should be low on your list of learning priorities
i would instead focus on understanding different shell options, learning the different kinds of input an output re-directions, learning about job control, and understanding how to avoid pitfalls related to parameter expansion, word splitting, and quoting
and learning to use and love printf instead of echo
also arithmetic expansion!
all of those things imo are more useful and take precedence over trying to do string processing without the aid of an external program
the whole point of bash is combining diffraction utilities together so learning awk is some form of learning bash
also fair i guess
imo awk and sed are deceptively simple tools and probably deserve their own learning paths
for context, i love awk and i love that you used it as your solution
it's a shame that it gets relegated to one-liners, it's quite a nice little programming language on its own
yeah, im actually a contractor and my primary strengths are aws and python so i do lean on those a lot but its a bit embarrassing when i ask colleagues for really simple things in bash if i need it, like i need to append an already really long script
im going to spend a lot of time this weekend and hopefully winter just getting better with it
in that case i'd definitely avoid worrying about the string processing stuff and focus on the things i listed above
some of my colleagues recently showed me a script they use that was clearly written by someone who was experienced with powershell and arrogantly assumed that bash worked the same as powershell
it was a mess
trust me that string processing was low on my list of "wtf" issues
i was planning on just reading the LPIC-1 book as my linux knowledge in general is lacking
admittedly i haven't read that myself. i've just spent a lot of time working with shell scripting and have spent more time than i care to think about reading the manuals
i learned a bit of bash first, then i spent a long time learning zsh, and thereafter spent some time learning posix scripting for compatibility across systems. i still know zsh much better than i know bash.
wonder how i'd do on this certification ๐
I use zsh in my terminal but i dont know it nearly well enough to understand the nuances
oh my zsh is all ive ever used
a book is usually good, however some books don't explain the "why" and just show lots of examples. that's not helpful
yeah if you have any good book examples that would be good
LPIC is highly regarded but tbh its a little dry too
honestly i don't. what i used to do (when i had more free time and motivation to do such things) is browse the manual and focus on learning one section of it at a time.
so i'd look through it until i saw something that seemed like it'd be useful, then i read it in detail, and wrote some programs to practice using it
that, or i'd come across something in someone else's code or a stackoverflow answer or whatever, and do the same
you learn a lot that way, albeit gradually
Yeah i wish i had that sort of time, i definitely think i need some structure to get through it effectively ๐ฆ
Thanks anyway!
Have you seen r/linuxupskillchallenge on Reddit? Basic but good.
I haven't no ill take a look thank you!
This looks fun thanks! @ember quiver
If you're moving a python script and some yaml files to a docker image wheres an appropriate place to put them in the filesystem? etc or usr/bin? or just the top level root
just make a /app directory or something and copy them in there
Top (The GNU C Library)
https://john-millikin.com/unix-syscalls @shell elbow
hi guys
im trying to host my website in a vps , i m connected with my vps using sftp
but when i type lls command it gives me this error psftp: unknown command "lls"
so any idea how can i fix it?
Bde#5vj%9ak0
Is there anyone available to help
it depends on the question.
Also see #โ๏ฝhow-to-get-help to increase your ods
also depends on the time
If that was a password you may want to change it ๐
I've never used psftp but it's apparently not the same as normal sftp: https://superuser.com/questions/859789/how-do-i-print-the-contents-of-my-local-directory-in-psftp
heyy i cant move a folder from home to etc in kali
can anyone help me out
do you get any error?
and are you using sudo?
ahh no i was not doing it in terminal
i was just drag dropping it
btw the problem was i was trying to drop it into the folder directly
so i had to open the destination folder in root first
Will give more Disk Space (it will, but also will do something else(
BAD
why
please delete your message
there are plenty beginners here who might try the command out
ok
For anyone who doesn't know: https://itsfoss.com/sudo-rm-rf/
i can try that without fear...
...as long as first commands before that
docker run -it --rm ubuntu:20.04 sh
or
vagrant init ubuntu/focal64
vagrant up
vagrant ssh
This may be silly but I have a bunch of gzip data. When I unzip it using terminal or by clicking, it yields a unix executable. The actual data is supposed to be jsonlines. curious if anyone else has had this issue.
Hello there,
I've a grub issue :"minimal Bash like line editing is supported" on a dual boot win11- fedora 36
Someone deleted one partition yesterday, and nothing work ๐ฌ
"Someone"? Who else is using this system?
A friend, he took my computer for his studies
And your "friend" deleted your partition? You can try restoring it but good luck
Fwsetup in the grub + put win part first
I solved it
Only UX part, win was clean
UX is not at abbreviation for "unix"
in english, UX usually stands for "user experience"
its from a trusted source but I hadnt thought of the security, thanks for pointing it out
My bad, thought UX stand for unix
Hi, I have a value ENV=DEV in .env file. I want to update it to ENV=TEST. I want to update it using terminal. Could anyone help me? Thanks in advance
I have to do it in my Bitbucket pipeline. So update has to be done in terminal.
is this a password? if it is you should delete it
well, change it actually
I've run a vps ubuntu 20. I was restarting my vps and it doesnt allow me to access through putty. When I check on my VPS system log. It ask me to log in. When I check on the qemu console. It said GRUB_FORCE_PARTUUID set, attempting initrdless boot.
The last thing I do on my vps was creating a new account to test the steamcmd on linux.
Which operational system do use to get virtualization?
and what do u use for virtualization exactly?
KVM. I'm using Massive KVM from ramnode.
What is your main OS?
Ubuntu 20.04 LTS
okay, from your nonsense answers i assume u a windows user ๐
anyway, i remember putty has some different standard regarding keys, i don't remember the difference
just install git bash, it has inside of it normal ssh / ssh-keygen inside as far as i remember
then u can pretty much generete if necessary ssh-keygen, RSA key and use its public key to create server, or connect by password if possible
ssh root@ip
optionally flag -i can provide some specific ssh-private key
it will make a more simple smooth experience
I just notice there's an abuse report report toward my VPS :(. Someone brute force my vps throug vps and from the team just stop my vps.
don't worry. That's pretty much normal... any cloud provider is constantly... bombarded with hack-scans.
For this reason people usually use ssh rsa ways only to connect and disable passwords
also for this reason they create servers only in private networks, that have cloud level firewalls that expose only necessary ports for services... (this feature is not available for too simple providers)
Rest assured, they scan also anything u will leave open ๐ Redis, Proxy servers, Databases. Be sure, no less than few days will pass as it gets hacked if u will fuck up xD
sudo rm -rf / haha
So
(Fedora)
Anyone know what services they are and how to disable them so they dont start on boot
i am not sure about 1st howver
2nd is dbus ie service for inter process cominication
3rd pulseaudio-pipewire it is emulation of โoldโ audio system running on top of
4th service pipewire it is used for audio/video
5th is gnugpg it is used for cryptography
you shouldnโt disable any of those
Guys, does anyone know how i can point a command to a script which then executes the true command?
for example i want to redirect ls to a script which runs ls then another command
but i still want to use ls as the command
I think you can do bash -i wrapper-script.sh command. I'm not 100% sure but I think that's how I solve entrypoints with extra level in Docker
can u describe somehow more understandable what u wish
requirements are unclear
get examples
just run the "true" command inside the script
if you want to replace an existing command, write a shell function
e.g. in bash
function ls() {
printf 'I am executing the "ls" command!\n'
command ls "$@"
}
ls -lA .
the function with override the ls "command", then you run the real command with the command builtin, which bypasses functions
Bash Reference Manual
note that command is not "portable" across POSIX shells, but it's available in Dash, Bash, Ksh, and Zsh which covers pretty much any shell you'd find outside of a retrocomputing museum or a Busybox environment
i have a question, when doing sudo apt update, how do i remove these warnings and errors?
You've got multiple things going on there and you may need to address them separately. I would start here: https://askubuntu.com/questions/1403556/key-is-stored-in-legacy-trusted-gpg-keyring-after-ubuntu-22-04-update
I get 3 warnings after Ubuntu 22.04 update on a Digital Ocean LAMP stack droplet.
W: http://repo.mysql.com/apt/ubuntu/dists/bionic/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/t...
What is Unix?
im trying to change my macOS' system date via python code and when i follow these steps nothing is changing on my system can anyone help? https://stackoverflow.com/questions/12081310/python-module-to-change-system-date-and-time
Did you try the last one on the list? If you don't mind the subprocess call it looks the most promising
FileNotFoundError: [Errno 2] No such file or directory: 'timedatectl'
ive overcome the problem by running an terminal command in the python script using os.system("sudo date .....")
thanks for your helping โค๏ธ
when i set up my linux, i set the language to german, now i changed it to english, can i change the [J/n] to [Y/n]?
are any of these safe
https://freevpshosti.com/vps-hosting.php
https://www.instafree.com/free-vps.php
i need a free linux vps
FreeVPSHosti , Get a free 2 gb ram managed vps hosting , get a free cloud hosting and get a free unlimited hosting cpanel without ads free vps hosting
free vps hosting no credit card
free vps hosting for students
free vps hosting trial
free vps hosting forever
free vps hosting lifetime
free vps hosting windows
free vps hosting server
free vps ho...
I'd sign up for AWS or Azure, and I think they give you a little machine for free for a month or something.
I wouldn't begin to trust some random site I'd never heard of.
linode is probably OK
oracle and salesforce too
SEO spam in the embed
clearly a trustworthy site
Is MacOS a good OS for programming in Python? Or programming in general?
๐
i would disagree with person above. Person above is Apple cult fanatic if he says so. Apple is good only for programming applications intended for IoS/Swift
For python it is the best to use Linux xD Rich development friendly open source ecosystem. No problems with CPU architecture, like u will have with Mac(Mac has arm64, while all servers and all other users Windows/Linux usually run on amd64). Free.
Linux is best if your target plaform is Linux servers/Linux desktop stuff. Which is for all python application essentially (be it Web dev or DS)
Linux can run containers (needed for Web at least) without virtualization shenanigans under the hood for secondary OS (like it happens with Windows/Mac)
Linux owner owns his OS fully and can do everything, best security xD U can increase your RAM memory through stuff like zRAM compressing data in section of RAM
Making yourself depended on propietary expensive solution is just silly. It has less support than Linux for development purposes. Only Linux supported first for all dev web tools
The thing is MacOS has much more available programs & apps. While Linux is good for programming, I believe thatโs it? And most importantly, MS Office
not sure which programs u mean.
For me using Mac will be the opposite, as u will need launching secondary OS (which will be Linux again) through virtualization in order to get access to necessary programs for development (after u will solve how to virtualize preferably amd64 instead of arm64 xD)
Also using Linux, u a making sure that your app is better developed/tested, less effort it made for deployment to Linux servers. It falls under DevOps Twelve Factor App 10nth rule about dev/prod parity https://12factor.net/dev-prod-parity
A methodology for building modern, scalable, maintainable software-as-a-service apps.
Perhaps do u mean that u will be forced to use programs like Safari Browser, which is more buggy laggi proprietary solution?
Internet explorer of Apple ecosystem, equivalent to how it was in Windows.
Microsoft eventually gave up this race and just uses Chrome engine (Blink) (Open source developed under direct Google control) under the hood in order to solve all rendering problems in its web browser Edge
There is a certain reason why Chrome dominates more than 70% of a market, while the rest percentage is some kind of Chromium forks xD
Exclusiveness of Apple stuff is just an illusion.
surely nice of them creating at least some competion to Google, which kind of already monopolized almost browser market though
to each his own opinion. Same thing i think about Windows. Windows is great only for development for Windows. Well, and certainly better access to games in Windows.
i am quite deep into web development, including wielding infrastructure as a code instruments for that. Only Linux can satisfy my needs for that.
it is wrong opinion when it comes for development for Linux as target platform (which it is in 95% cases in terms of web development). It is so at least according to Twelve factor app and DevOps engineering
it is quite obvious, that best development for Linux is made from Linux
https://betanews.com/2022/10/02/spotify-is-forcibly-installing-on-windows-10-and-windows-11-systems/ what can be worse than proprietary systems making decisions for you? xD
Sure. That's why we use Managed deployments like AWS ECS or Managed Kubernetes Clusters
Turning the problem of security of individual OSes almost completely delegated to cloud provider, or turning security problems to... more centralized controlling issue
Nevertheless it again becomes Linux to run it... Because it is scalable from being free (and from minimal overhead), and from completely being configurable (as u mentioned) to be turned secure. And the most importantly in comparison to other OSes it is easly automated in its configurations. That's the most important part too
Linux can be easily stripped to almost zero, to make smaller possible attack surfaces
Shrugs. It is more convenient to run dev env on current machine, then accessing it with virtualization/remote accesses.
Makes less complicated instructions invokable directly from current machine
And less performance load to machine / less networking delays
Plus running Linux as your OS, u get full access to its full priveleges, like increasing ram through zRAM xD, or easily ability to launch its GUI parts if necessary
Also it is more secure to leave your PC at work / other places, when u know it its file system is completely encrypted with LUKS (which is turned in one checkbox during linux installations) xD
They aren't disadvantaged when they don't deal with development for Linux. They are disadvantaged when they are developing for Linux.
Developing for Linux makes software immediately tuned it for that, less development hassle to tune stuff.
Easy. Developer in frontend framework React.js, developed his stuff in Windows. His javascript library imports were case insensitive in Windows due to its case insensitive file system. When we tried to compile it for Linux, it immediately failed because of this reason. (extra capital letter where it was not needed)
Same problem with also compiling any stuff essentially, or having software that uses OS internal features (including its things how its filesystem works)
This problem is moderately low for interpreting language, but than lower to hardware you go, then more troublesome it will become
All right. Software in Linux including Python has more parallelism options ;b in Linux multiprocessing through fork is available, while in Windows it is not.
Small differences one by one, make difference
i say that multiprocessing through function called fork is not working on Windows
// Windows Subsystem for Linux (WSL) - previously Bash on Windows
// - Example: `C:\\Windows\\System32\\wsl.exe`
//
// Git-bash on Windows
// - Example: `C:\\Program Files\\Git\\bin\\bash.exe`
//
// Cygwin
// - Example: `C:\\cygwin64\\bin\\bash.exe`
which of these is the least miserable to use? ||i could also try learning powershell||
you can use powershell in linux
wsl is overall the best because it is just VM and thus offers best compatibilty. not everything runs on cygwin and git bash
How can I do this?
Modify the executable.sh file again, add the command for it to create the following directories upon execution.
thanks for the thoughts went with wsl 
very diplomatically phrased, Sir/Ma'am
definitely WSL if you can get it working.
Have a nice day everyone! I was trying write sh file for installing. I did create virtualenv. But ฤฑ can't access this. These the codes ```py
python3 -m venv venv
source ./venv/bin/activate
I get the err
```code
install.sh: 5: source: not found
``` How can ฤฑ fix this
Not sure if this is the right path, but what shell are you using? source is a Bash construct, it doesn't exist in standard sh and I've been caught out with this before in other contexts
Actually ฤฑ dont know how to use. I want do dynmic systm for installing project. I havent enough info on bash/ linux. Have u suggestions for me?
Are you running the script manually or is something running it automatically? And what operating system are you using?
And do you have a shebang at the top of your script?
https://linuxize.com/post/bash-shebang/
Guys i need help in an exercise
is there any way somone can help?
Is there a way to set a permanent $PYTHONPATH while being in a Conda environment?
on UNix, I'd expect this to work
$ PYTHONPATH=/path/to/python conda
i work in vscode, i set variables in .vscode/settings.json file for dev env without docker
or at docker-compose level when in docker dev env
when project is built into docker, permament ENV can be defined at Dockerfile level as ENV var
This sounds promising, i will check this out
how to get env variables?Python-dotenv could not parse statement starting at line 1
ask in a help channel #โ๏ฝhow-to-get-help
sorry but currently it is closed
Oh btw, forgot to realize. If u need to touch PYTHONPATH, they u a doing something horribly wrong already very highly likely
So better tell why u came to need changing it in the first place in detail
please help it's showing me error
No i have a user on the hpc and i have to "manage" everything myself, so most likely i did the mistake. I just cloned the repo, created a conda environment and started working. ๐ Just hast du change PythonPath so the modules got recognized for pytest
Owner of code did not know how python Packages really work then.
Recommendation to move packages code to start from root folder of a project, and ensuring no __init__.py is present in root folder of a project
they will be recognized as packages without PYTHONPATH hacks then
Hii guys I need help on something..So is there any way I can use CURL to not completely download the html page but till the I find this particular phrase..suppose I have a page with app I'd and its status..so instead of loading the whole page if it finds this app I'd and its status I want the curl to terminate...I am having a problem because sometimes curl takes a lot of time return the html page so I want to a way to hack a way around
i think u a possibly misunderstanding what u a wishing to have
u a wishing only partially rendered output to stdout perhaps instead of partial download
anyway, doing partial http request download is black magic of very low level... not really feasable put amount of effort for low gain. It requires super high level if it is possible.
Okay thanks.. I have a yarn and spark running and sometimes a job is shown completed in spark but not in yarn..so using curl we want to check in yarn server if it's active or not..but some times it takes to much time to load..so we want a partial one so that we can kill the jobs
what takes too much time to load, job result, or list of all jobs?
- Anyway, i would have tried finding solution, how to change request to your Spark in the way that it sould be reporting only job status
- If u need to see those statuses only manually, then perhaps u are missing some observability (monitoring/logging) solution which would have showed in some nice interface this stuff (technically automatable too to extract this info with script out of monitoring solution though)
Has anyone programmed a drone with python? If so can you please tell me what drone you ordered
Hi all, is there a chance to get link for ubuntu discord server ?thanks
Many thanks
Arch Linux + KDE Plasma
rc=-1908 error code
this happened to me the other time and idk how i fixed it
does anyone know the solution to this? installed virtualbox from the AUR
did you read the error message and do what it suggests?
I've encountered that error on Debian testing and needed to reinstall DMKS and/or the VirtualBox DMKS package (along with kernel headers, etc.).
I'm assuming you followed the instructions in the error message already...