yup
#esoteric-python
brisk zenith
brisk zenith
yup :D
just try linting it example.py:6:5: F821 undefined name 'cout' example.py:6:31: E703 statement ends with a semicolon example.py:7:5: F821 undefined name 'cin' example.py:7:12: F821 undefined name 'x' example.py:7:13: E703 statement ends with a semicolon example.py:9:5: F821 undefined name 'cout' example.py:9:28: E703 statement ends with a semicolon example.py:10:5: F821 undefined name 'cin' example.py:10:12: F821 undefined name 'y' example.py:10:13: E703 statement ends with a semicolon example.py:12:18: F821 undefined name 'x' example.py:12:27: F821 undefined name 'y' example.py:13:5: F821 undefined name 'cout' example.py:13:13: F821 undefined name 'x' example.py:13:27: F821 undefined name 'y' example.py:13:51: F821 undefined name 'endl' example.py:13:55: E703 statement ends with a semicolon
rare juniper
brisk zenith
thank you <3
rare juniper
I wish I was bigbrain enough to know what the heck this code was doing. I was surprised you weren't overriding any dunder methods (except __code__?) to do stuff like cin >> x;
brisk zenith
__code__ is all you need really
it contains the bytecode of the function
which can be modified
to turn cin >> x; into x = input()
etc.
rare juniper
So does exerting control over that just give you the ability to create literally any syntax you want?
rare juniper
Ohhh
brisk zenith
it simply does cin.__rshift__(x)
rare juniper
So it has to compile to SOME kind of identifiable bytecode pattern, and then you replace that with the one you want?
brisk zenith
yes
rare juniper
Neat!
Is this done on a level that you could do stuff with type hints, or does that not show up in the bytecode?
brisk zenith
that doesn't appear in the bytecode
rare juniper
Makes sense. Ahhhhh, esoteric python. If only my homework were this interesting...
>>> one = 1
>>> two = 2
>>> three = 3
>>> one, two, three
(1, 2, 3)
>>>```
I wonder if you could override this comma behavior in an object...it's not the object's behaviour
it's part of the syntax
in fact, i'm pretty sure it triggers the bytecode for tuple creation
rare juniper
Oooh
brisk zenith
>>> def x():
... a, b, c
...
>>> dis.dis(x)
2 0 LOAD_GLOBAL 0 (a)
2 LOAD_GLOBAL 1 (b)
4 LOAD_GLOBAL 2 (c)
6 BUILD_TUPLE 3
8 POP_TOP
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
>>>
notice the BUILD_TUPLE instruction
wind maple
do u know what's real funky
rare juniper
wad
brisk zenith
a tuple that contains itself? :^)
rare juniper
that seems reasonable
>>> foo = (1, 2)
>>> [x, y] = foo
>>> x
1
>>> y
2
>>> x, y
(1, 2)```the [] aren't necessary
rare juniper
right
wind maple
yeah
brisk zenith
i guess it's just weird syntax
rare juniper
Hmmm
brisk zenith
probably the same bytecode
wind maple
I wonder if it's in the spec or if it's a cpython thing
rare juniper
It lets you do multilines tho, which is cool! (I assume you can't without the list)
>>> [x,
... y,
... z,
... a,
... b,
... c] = (1, 2, 3, 4, 5, 6)
>>> x
1
>>> y
2
>>> z
3
works as well with a tuple
rare juniper
Yeah, I'd probably use a tuple to get that behavior (e.g. multiline imports)
rare juniper
I'm guessing it has to-- YUP
wind maple
yep
rare juniper
I was gonna say, has to do with unpacking
>>> print(*[1, 2, 3])
1 2 3
>>> print(*(1, 2, 3))
1 2 3```was my guess
rare juniper
nah?
wind maple
unpacking works on any iterable
rare juniper
ah true
wind maple
you can't have any iterable as the left hand side
rare juniper
But this assignment wouldn't wouldn't work on some
wind maple
only lists and tuples
rare juniper
otherwise it'd just override the entire thing
brisk zenith
it should allow for sets on the left hand side to make it so the assignment is "randomised" :^)
rare juniper
>>> {x, y, z} = (5, 6, 7)
File "<stdin>", line 1
SyntaxError: can't assign to literal``` bummer>>> x, y, z, a, b, c = random.sample([1, 2, 3, 4, 5, 6], 6)
>>> (x, y, z, a, b, c)
(2, 5, 4, 6, 1, 3)``` not esoteric python but quickest thing I could come up with to get a similar syntax. Wonder how you'd do it if you really wanted to use set order "randomness"¯_(ツ)_/¯
to be fair, i'm pretty sure using sets with integers like that would just put them in order
because integer hashes are just the integers themselves
apart from -1 which hashes to -2 i think
actually, it wouldn't be in order
rare juniper
Random question,
I'm interested in making a tool that transpiles Python into Golang (requiring you to fill in types as necessary, or just guessing). The intention is to create a learning tool that will eventually live in a Golang IDE.
I originally imagined parsing the actual strings of code, but would it make more sense to try to generate code starting from Python bytecode? Or even both?
https://github.com/google/grumpy is an old project by Google that does something like this (but not in your IDE) for Python 2.
Well, actually
I'm just wondering how you'd approach the problem -- not necessarily asking you to make any verdict on what the best possible solution is haha
brisk zenith
you'd have to break down the python source into an AST (which can be done using python's ast module), then modify the AST to make it fit with go's logic, then convert that AST into go source code.
easier said than done, of course.
that's how i would do it, at least.
but i don't know much about this sort of thing
cunning wave
you could also go one step lower and translate the bytecode
rare juniper
Yikes, yeah... Nah, that's helpful. I'm still debating on whether I want to try reinventing the entire thing from scratch, or forking (or even just writing a wrapper for) this grumpy thing to help it live in my IDE.
The former would definitely be more educational
brisk zenith
literally the only thing i know about go is that go 2 has generics which would probably come in handy
i seriously don't know a single other thing about go though
like, at all
rare juniper
Long story short it made the code really verbose and ugly. Not what I'm looking for at all.
I'm just looking for something that gets "pretty close to code that will compile"
not a perfect transpiler for Python that churns out really ugly code with a lot of framework to make sure everything runs perfectly
Almost like a phrase book for people that speak Python and want to tour Golang.
"How do I say thank you in Spanish? -> gracias
"How do I say print('Hello world') in Go?" -> fmt.Println("Hello world")
rare juniper
Been playing with the Python to golang thing. My progress so far has been really hacky, but
import unittest
from pygotranslate import py2go
class TestPrintStatements(unittest.TestCase):
def test_single_quotes(self):
def main():
print('Hello, world!')
go = """
fmt.Println("Hello, world!")
""".strip()
self.assertEqual(py2go(main), go)
def test_double_quotes(self):
def main():
print("Hello, world!")
go = """
fmt.Println("Hello, world!")
""".strip()
self.assertEqual(py2go(main), go)
def test_nested_quotes(self):
"""
The program should automatically escape quotation marks in the case of
converting single quotes to double quotes or vice versa.
"""
def main():
print('Quotation mark: "')
go = '\n'.join([
'fmt.Println("Quotation mark: \\"")'
])
self.assertEqual(py2go(main), go)
class TestVariableStorage(unittest.TestCase):
def test_string_store(self):
def main():
cake_opinion = 'delicious'
print('Cake is', cake_opinion)
go = '\n'.join([
'cake_opinion := "delicious"',
'fmt.Println("Cake is", cake_opinion)'
])
self.assertEqual(py2go(main), go)
if __name__ == '__main__':
unittest.main()
I got it to pass these tests.
(Actual code above in link)
rare juniper
I went with the first thing I could figure out from Googling. I was originally doing regex
It's hilarious to me that bytecode is literally easier than regex
rare juniper
wow wrecked
brisk zenith
!e ```py
<redacted code to avoid the spammies>
i will yeet you out of existence
!e ```py
import ast
code = "for i in range(10): print('hello')"
for thing in ast.walk(ast.parse(code)):
print(thing)
night quarryBOT
@brisk zenith Your eval job has completed.
001 | <_ast.Module object at 0x7f2294cfbf60>
002 | <_ast.For object at 0x7f2294d91240>
003 | <_ast.Name object at 0x7f2294d1d198>
004 | <_ast.Call object at 0x7f2294d1e080>
005 | <_ast.Expr object at 0x7f2294d1e128>
006 | <_ast.Store object at 0x7f22954684e0>
007 | <_ast.Name object at 0x7f2294d1e0b8>
008 | <_ast.Num object at 0x7f2294d1e0f0>
009 | <_ast.Call object at 0x7f2294d1e160>
010 | <_ast.Load object at 0x7f22954683c8>
... (truncated - too many lines)
Full output: https://paste.pythondiscord.com/kelivehijo
brisk zenith
breaks the code up into separate chunks
bytecode seems like it would be a big pain to work with for a first attempt
i mean, feel free to give it a go. :D
just be aware that there's other options if you get fed up
rare juniper
Trying to figure out how I can use this to do what I was doing with bytecode... hmmm
rare juniper
import ast
code = """print('hi', 3, True, lol, print)"""
for thing in ast.walk(ast.parse(code)):
if thing.__class__.__name__ == 'Call':
print(thing.func.id)
for arg in thing.args:
if arg.__class__.__name__ == 'Str':
print(arg.s, 'Str')
elif arg.__class__.__name__ == 'Num':
print(arg.n, 'Num')
elif arg.__class__.__name__ == 'NameConstant': # Awlays booleans or what?
print(arg.value, 'NameConstant')
elif arg.__class__.__name__ == 'Name':
print(arg.id, 'Name')
else:
print(arg.__class__.__name__)
print(dir(arg))
out:
hi Str
3 Num
True NameConstant
lol Name
print Name```I'm not sure if this is better yet... Maybe.
Yeah, I think it is. Just need to learn more about it.
rare juniper
I got to about the same point -- maybe a bit further, I can do stuff like
a, b = 3, 2
x = 1, 2, 3
y = "Hello", "World"
print(a, b)
print("Hello world!")
``` to get
```go
a, b := 3, 2
x := [3]int{1, 2, 3}
y := [2]string{"Hello", "World"}
fmt.Println(a, b)
fmt.Println("Hello world!")
``` with `ast`
It's pretty cool, although I'm starting to rethink my approach of building lines by iteration (be it iteration of bytecode or otherwise). I should probably try to build it as a tree and join everything together in the end...
I should've definitely tried leveraging the greentreesnakes docs more than I did writing it.Glad to see it includes annotations... Could use them to help this along...
UGH, now that I actually have the means to do this I'm gonna want to actually do it. Bummer, I planned on having free time later this year.
grave rover
Try the ast.NodeVisitor instead
vocal oyster
!e
def pyramid(n):
return "\n".join(" ".join(str(i + i * j) for j in range(i)) for i in list(range(1, n + 1)) + list(range(n - 1, 0, -1)))
print(pyramid(5))
@vocal oyster Your eval job has completed.
001 | 1
002 | 2 4
003 | 3 6 9
004 | 4 8 12 16
005 | 5 10 15 20 25
006 | 4 8 12 16
007 | 3 6 9
008 | 2 4
009 | 1
@grave rover luckily I did haha
It's like writing a spider with scrapy except for code
grave rover
Meanwhile I need to break my brain with math
Finally got the derivative of a func
Forgot the product rule
rare juniper
Posted a bit more progress in the general chat last night. Oh product rule is ez
"I preferr to be called babe over baby"
Oh wait
Well that's my shortcut for the quotient rule lol
(BA'-AB')/BB
(I actually say bahb over beebee but this is where it comes from)
fallen heath
I know some of these symbols
grave rover
n over i is a constant so don't need to derive that
Pi is a constant too
(1-t)^(n-i) is f(x)
t^i is g(x)
rare juniper
Ohh dang that's formal
fallen heath
What function is that image?
rare juniper
I was thinking
I might want to actually consider forking https://github.com/berkerpeksag/astor/ for my Python -> Go translator
It reverses an AST back into Python -- maybe it'd be easier to fork that to reverse an AST into Golang, than to write all that logic from scratch
I'm gonna call it Goast
Someone LITERALLY already came up with this name before me???
honest juniper
name it Ghost then
Ghost (go-ast)
ghost is better than goast to me
more polished
I'd make two versions, one for 3.8 and 3.7, python AST is going to be removing some AST literals in 3.8
rare juniper
Good to know. I did think they were a bit silly
especially namedconstant
that one is annoying
"yeah it's a bool or None"
Maybe since I wanna fork https://github.com/berkerpeksag/astor/blob/master/astor/ to do this
I could call it "goaster"
or "ghoster" -- one rhymes with toaster
:^) (well they both do but..)
grave rover
@fallen heath it's Bezier
def f(i, n, t):
return (1-t)**(n-i)
def f_d(i, n, t):
return -1 * (n - i) * (1-t)**(n-i-1)
def f_dd(i, n, t):
return (n - i) * (n - i - 1) * (1-t)**(n-i-2)
def g(i, n, t):
return t**i
def g_d(i, n, t):
return i * t**(i-1)
def g_dd(i, n, t):
return i * (i-1) * t**(i-2)
def l(i, n, t):
return f_d(i, n, t) * g(i, n, t)
def l_d(i, n, t):
return f_dd(i, n, t) * g(i, n, t) + f_d(i, n, t) * g_d(i, n, t)
def r(i, n, t):
return g_d(i, n, t) * f(i, n, t)
def r_d(i, n, t):
return g_dd(i, n, t) * f(i, n, t) + g_d(i, n, t) * f_d(i, n, t)
def b(i: float, n: int, t: float) -> float:
return nCr(n, i) * f(i, n, t) * g(i, n, t)
def b_d(i: float, n: int, t: float) -> float:
return nCr(n, i) * (l(i, n, t) + r(i, n, t))
def b_dd(i, n, t):
return nCr(n, i) * (l_d(i, n, t) + r_d(i, n, t))
b(t), b'(t) and b"(t) implemented as readable as I could
snow beacon
In order to manipulate them programmatically, you'd want to put those functions into a dict of lists that represents what is the derivative of what, right?
grave rover
ended up just using numpy.gradient instead
sick hound
[print((i+' shark'+" doo"*6+"\n")*3+i+" shark!")for i in"Baby Daddy Mommy Grandpa Grandma".split()]
```baby shark
candid cobalt
doo doo doo - 3 chars shorter
for i in'Baby','Daddy','Mommy','Grandpa','Grandma':print(f'{i} shark{" doo"*6}\n'*3+i+" shark!")
for i in'Baby','Daddy','Mommy','Grandpa','Grandma':print(i,f'shark{" doo"*6}\n'*3+i,"shark!")```A little shorter
tropic gulch
grave rover
Delete this
snow beacon
When you think about it, a shark is just an esoteric version of a python.
autumn moss
when you really think about it....
a python is just an esoteric version of a lua
🤔
honest juniper
not true
python came before lua
just because they're both written from C doesn't make them the same
snow beacon
Python is just assembly with syntactic sugar
brisk zenith
can that really be said for interpreted languages?
humble tartan
at the end of the day assembly is still somewhat interpreted (at least for x86) since a good chunk of your instructions will be split into μops :^)
cunning wave
every assembly is interpreted, the action of a computer understanding for example what a MOV operations means and then executing it can also be counted as interpreting
cunning wave
its definitely not
fast torrent
@cunning wave sniff... Do I smell someone talking about rust without me there!
cunning wave
since when are you into rust
brisk zenith
since when are you
cunning wave
i thought youre the go type of person
since when am i
i can tell you very precisely
brisk zenith
please don't
tropic gulch
doxx
fast torrent
@brisk zenith @cunning wave When are we gonna stop shitposting and remember that you are on the sharpbot team? 😠
cunning wave
how is that shitposting
also i only responden because of the ping i have this very fun 8 page document about city geography to learn for tomorrow
fast torrent
Yummy
cunning wave
and then tomorrow learn for physics on tuesday and THEN i can start learning for maths and THEN i can start learning for my A levels and THEN i am done with school
today i learned that Barrios Cerrados and Ciuadades Valladas are basically names for the same thing but one is inside the city the other isnt. And now comes the interesting question
Why do i have to know the different spanish names for gated communities inside and outside south american cities
@fast torrent explain that in a reasonable way to me please
cunning wave
but how am I, the average european citizen ever gonna need to know whats fancy language for gate community inside countries i normally dont visit
snow beacon
When you achieve your lifelong goal of becoming the Brazilian diplomat, they might catch you decoding their Python code they compressed into as few characters as possible, and you'll have to flee to a safehouse in Argentina. But all you heard from your Spanish informant is "Está en el barrio cerrado al norte" as he runs off to distract the pursuers. You'd need to know whether it's in or out of the city.
rare juniper
-deletes message- whoops wrong chat
tepid glacier
Is there any way to modify a dunder method of a builtin class without modifying cpython's source directly?
Kinda like how forbiddenfruit does it with normal methods
want to be able to do
>>> import abomination
>>> some_list = [1, 2, 3, 4, 5]
>>> some_list[0]
Getting an item!
1
i've done that before. i used ctypes to patch the method in memory, i think
lemme check
actually no, i had to patch the entire class i think
let's see.
can't check because gitlab needs 2fa and my phone's dead and i can't be bothered to charge it :D
tepid glacier
rip
tropic night
I thought the c-builtins was immutable
whole kiln
They are but I guess that doesn't stop some people
tropic night
so, how would you? monkeypatch the mro?
whole kiln
I don't know myself. Just read about the ctypes memory patch so I thought it was possible
tropic night
I'm kinda betting you can't do it
But let's see what @brisk zenith can come up with
whole kiln
I dunno, patching memory sounds kinda promising, even if changing built-ins is by design prohibited.
brisk zenith
i've patched the memory of the globals dict before so its definitely possible.
calm rampart
I wonder how far you could get just by setting the Py_TPFLAGS_HEAPTYPE bit
the other problem you have is saving the original values
you can't just save foo.__bar__, that's a slot wrapper that AIUI doesn't contain a reference to the actual function
I suspect the major place where it would blow up is if you try to remove a method and it tries to deallocate a subtable that wasn't allocated on the heap
calm rampart
ok, no, heap types have them all allocated in a row, different from standard types... you'd have to mess with all the slots individually
tropic night
Ahhh.. the dict, sounds like kittens would die from that.
tepid glacier
yeah
It's a horrible idea but I was curious to see if I could implement haskell-style list builtins
@tropic night Yeah forbiddenfruit gets past that by replacing the builtin with PyDict_SetItem()
glacial rampart
(0for[]in[]) can y'all come up with a shorter generator expression?
glacial rampart
It doesn't have to execute successfully (list(your_gen) may fail)
frozen fog
(0 for _ in [])
One char less?
frozen fog
oh i see. The lack of whitespace is intentional. Neat. Didn't know that was valid.
glacial rampart
if you ever need to raise inside of an eval(), (0for[]in[]).throw(Exception)'s got your back 
brazen geyser
class DivisibleSequence:
def __truediv__(self, value):
return self // value
def __floordiv__(self, value):
return tuple(
self.__class__(
self[int(i/value*len(self)):int((i+1)/value*len(self))]
) for i in range(value)
)
def __mul__(self, value):
print(value)
if isinstance(value, int):
return self.__class__(super().__mul__(value))
elif isinstance(value, float):
div, mod = divmod(value, 1)
return self.__class__(self*int(div) + self[:int(len(self)*mod)])
def divisible(sequence):
class DivisibleSubtype(DivisibleSequence, sequence.__class__):
pass
return DivisibleSubtype(sequence)
print(divisible((2, 3, 4, 5)) / 2)
print(divisible('1231231') * .5)
print(divisible('345234') * 2.5)
who wants to tell me what a monster i am
snow beacon
You're a monster.
By the way, it's probably easier to just do exec("raise Exception")
Depending on which quote marks you used.
brisk zenith
@glacial rampart i do that in my massive one-liners, it's a neat trick.
tepid glacier
Came up with this a couple days ago for a golfing challenge
import os,sys
a,f,*y={*sys.argv},lambda x,*i:''if not{*i}&a else x.format((a&{*i}).pop()),'arch','gentoo','nixos','opensuse','kali','void','bedrock','slackware'
os.system(f"neofetch {f('--ascii_distro {}',*y)}{f('_{}','small')} {f(f'--ascii $(cowsay btw i use '+f('{})',*y),'cow')}{f('|lolcat','rainbow')}")
Spreading the most important of stories I see.
sick hound
We just wrote this py (lambda a:setattr(a,'d',((lambda b:b(b))(lambda b:lambda f,x:(lambda c,d:lambda:c(*d))(b(b),(f,f(x)))))(lambda n:print(n)or n+1,0))or list(setattr(a,'d',a.d())for _ in iter(int,1)))(type('',(),{})())
With no bytecode madness or imports, that code creates an infinite loop and prints a lot of numbers
wind maple
seems unnecessarily verbose
doesn't for i, _ in enumerate(iter(int, -1)): print(i) achieve the same
sick hound
yes, but that's a statement, we wrote a lambda-expression-thing
rare juniper
return True != False != False
0 LOAD_CONST 1 (True)
2 LOAD_CONST 2 (False)
4 DUP_TOP
6 ROT_THREE
8 COMPARE_OP 3 (!=)
10 JUMP_IF_FALSE_OR_POP 18
12 LOAD_CONST 2 (False)
14 COMPARE_OP 3 (!=)
16 RETURN_VALUE
18 ROT_TWO
20 POP_TOP
22 RETURN_VALUE
return True != (False != False)
0 LOAD_CONST 1 (True)
2 LOAD_CONST 2 (False)
4 LOAD_CONST 2 (False)
6 COMPARE_OP 3 (!=)
8 COMPARE_OP 3 (!=)
10 RETURN_VALUE
return (True != False) != False
0 LOAD_CONST 1 (True)
2 LOAD_CONST 2 (False)
4 COMPARE_OP 3 (!=)
6 LOAD_CONST 2 (False)
8 COMPARE_OP 3 (!=)
10 RETURN_VALUE
return (True != False) != (False != True)
0 LOAD_CONST 1 (True)
2 LOAD_CONST 2 (False)
4 COMPARE_OP 3 (!=)
6 LOAD_CONST 2 (False)
8 LOAD_CONST 1 (True)
10 COMPARE_OP 3 (!=)
12 COMPARE_OP 3 (!=)
14 RETURN_VALUE```I don't understand the top one
snow beacon
A != B != C is equivalent to A != B and B != C
tropic gulch
If you imagine the call stack, I assume this might be what it looks like after each operation:
0: True
2: True False
4: True False False
6: False True False
8: False True # popped and compared top two, pushed result
10: False
12: False False
14: True
(just guessing tbh)
snow beacon
Wouldn't 8 push True?
tropic gulch
hmm... right
snow beacon
True != False, so it pops rather than jumping
snow beacon
The JUMP_IF_FALSE_OR_POP functions like an and operator, and the duplication and rearrangement of constants is because Python needs to use the same value, False, for both halves of the and clause.
scenic aspen
while os.fork()+1:pass
gilded orchid
Are there any resources available to learn stuff like this, it seems really interesting
scarlet osprey
It's just a load of nonsense
You'll pick it up from using Python and doing lower level stuff
sick hound
@tepid glacier yes there is a way
monkey patching internal cpython methods at runtime
tepid glacier
👀
sick hound
tepid glacier
:+1:
sick hound
Note: Only available for Posix compatiable systems
sick hound
👌
viral hedge
So to get funky, how would i nest lambdas where I convert a variable to an int, then check if it's in a specific range of numbers?
if str_.isdigit():
num = int(str_)
return 5 < num < 50``` kind of dealmight not need to nest lambdas there
lambda n: int(n) if n.isdigit() and 5 < int(n) < 50 else None
iirc int(n) will only be evaluated if n.isdigit() is True
In [2]: f = lambda n: int(n) if n.isdigit() and 5 < int(n) < 50 else None
In [3]: f('4')
In [4]: f('a')
In [5]: f('10')
Out[5]: 10
yep
sick hound
yup
but not that version
The master branch is on pypi
the v1-extended branch with hookify (cpython patcher) is just a demo for that can be done
rugged sparrow
(lambda x:x(x,(lambda x:[[[0]*len(x[i])for i in range(len(x))],x])((lambda a:(lambda x,y:[[[[x[r].pop(c),x[r].insert(c,'*')]if y[0][r][c]else[x[r].pop(c),x[r].insert(c,y[1][r][c])]for c in range(1,a-1)]for r in range(1,a-1)],x][1])([a*[0]for x in range(a)],(lambda x:[[[[[(lambda x,y:[y.append(x[1][r][c]),x[1][r].pop(c), x[1][r].insert(c,y[0]+1)])(x,[])if x[0][l][j]else 0for j in range(c-1,c+2)]for l in range(r-1,r+2)]for c in range(1,a-1)]for r in range(1,a-1)],x][1])((lambda x:[[[[x[0][r].pop(c),x[0][r].insert(c,int.from_bytes(open('/dev/urandom','rb').read(1),'big')/10>20)]for c in range(1,a-1)]for r in range(1,a-1)],x][1])([[a*[0]for x in range(a)]for x in range(2)]))))((lambda i:i if i>2and i<12else exit())(int(input('Board Size (max 9):'))+2)))))((lambda x,y,p=(lambda x:[print('\x1b[1;32m ',*range(1,len(x[0])-1),sep=' '),[[[print(f'\x1b[1;32m{r}\x1b[0m',end='')if c==0else[print(' ',end=''),print(['■','F'][x[0][r][c]],end='')if x[0][r][c]!=2else print(x[1][r][c],end='')]for c in range(len(x[0])-1)],print()]for r in range(1,len(x[0])-1)],x][-1]),i=(lambda f,i:[f[0][int(i.split(',')[0])].pop(int(i.split(',')[1])),f[0][int(i.split(',')[0])].insert(int(i.split(',')[1]),{'f':1,'c':2}[i.split(',')[2].lower()]),[print('You Lose'),exit()]if'*'==f[1][int(i.split(',')[0])][int(i.split(',')[1])]and'c'==i.split(',')[2].lower()else f][2]),q=[]:[p(y),q.append(i(y,input('Type Row,Column,[(F)lag or (C)lear]:'))),x(x,q[0])if(lambda y,q=[],w=[]:[[[q.append(y[0][r][c]==1and y[1][r][c]=='*')for c in range(len(y[0][r]))]for r in range(len(y[0]))],q][1].count(1)!=[[[w.append(y[1][r][c]=='*')for c in range(len(y[1][r]))]for r in range(len(y[1]))],w][1].count(1))(y)else print('You Flagged All The Bombs!')]))``` its minesweeper in one line. Ex: input is ``1,2,C`` which would select row 1 column 2 and clear ituses a modified Y combinator for recursion and passing the minefield
brisk zenith
oh spicy
tropic gulch
I still see a few superfluous spaces in there 😛
rugged sparrow
@tropic gulch i havent minimised it yet
tropic gulch
👌 all good
tropic gulch
😄
stray needleBOT
tropic gulch
there's probably still some more potential, but it's awful enough as it is already 😉
stray needleBOT
brisk zenith
now i know: use git mv to move files, got it.
stray needleBOT
brisk zenith
challenge 06 has arrived!
here's something a bit different. today's task is to make python believe that there's a fancy keyword called Maybe, which should basically act as an indecisive third boolean :D. this challenge may be quite tricky for some of you, but i'm sure the solutions will be fascinating! see the repository for more details before you get started: https://github.com/python-discord/esoteric-python-challenges/tree/master/challenges/06-maybe-keyword
should probably clarify that i'm not looking for anything as extreme as a cpython fork which adds an actual keyword or whatever. your implementation could just be a module, function decorator, or whatever else (as long as it's reasonable and does the job, of course!)
brisk zenith
not quite, int is a builtin function. True and False are keywords, and if is too.
(but the focus is mostly on True and False, as they have values)
brisk zenith
i have to sleep now, so i'll be here tomorrow if anyone has any questions :)
stray needleBOT
snow beacon
First!
weak plover
Question for the Maybe challenge, do all uses of the maybe block in the code have to have the same value?
brisk zenith
nope, should be random each time.
wind maple
(use random.seed if you wanna test it or something)
brisk zenith
for i in range(3):
print(Maybe)```
could output `True` then `False` then `False`, or something else entirely. randomise the value for each access of `Maybe` :D@slim echo i left a comment for you. :)
slim echo
Coolio
Yeah you have a point
It only really works correctly as an attribute of the module
brisk zenith
@slim echo yeah, that's fine though! you've made a good effort. when i tried to implement this myself, i didn't get it perfect either. there's always something that doesn't quite work haha
@snow beacon made a short comment on yours :D
brisk zenith
sure, feel free to make any changes.
slim echo
*interpreter errors lol
Interpreter exit errors squashed
(probably)
Did you try testing with it as part of the module or only as its own name?
brisk zenith
yup, that works pretty well. :D
slim echo
:)
I actually spotted IFcoltransG's sys.modules hack and decided to put it to use in making the module work
slim echo
Other than __del__ and __delattr__/__setattr__ I don't think there is a way to detect reliably assignment or deletion
(Also you aren't allowed to del True so I implemented that too)
brisk zenith
oh haha that's nice. yeah, preventing assignment (especially before the assignment even runs, like how True = False will raise an error before anything begins to execute) is pretty damn tricky without doing some nasty hacking, that's for sure.
slim echo
The module hack actually made it really easy
wind maple
I'm not gonna make a PR because I ran out of motivation but this is what I did
import builtins
import dis
import inspect
import random
import sys
class _Maybe(type):
def __bool__(self):
return bool(random.getrandbits(1))
def __str__(self):
return 'True' if bool(self) else 'False'
def __lt__(self, other):
return bool(self)
class Maybe(int, metaclass=_Maybe):
pass
sys.modules['__main__'].Maybe = Maybe
def check_ops(code):
for i in dis.get_instructions(code):
if inspect.iscode(i.argval):
check_ops(i.argval)
if i.opname in 'STORE_NAME STORE_FAST':
if i.argval == 'Maybe':
print(i)
raise SyntaxError('Cannot assign to Maybe')
class DumbassWrapper(dict):
def __init__(self):
for name, mod in sys.modules.copy().items():
if name in sys.builtin_module_names:
continue
try:
with open(mod.__file__, 'r') as f:
code = compile(f.read(), 'dicks', 'exec')
try:
check_ops(code)
except SyntaxError:
print(name)
except UnicodeDecodeError:
"???"
self.update(sys.modules)
def __setitem__(self, key, value):
with open(value.__file__, 'r') as f:
code = compile(f.read(), 'dicks', 'exec')
check_ops(code)
super().__setitem__(key, value)
sys.modules = DumbassWrapper()
original = builtins.__build_class__
def build_class(func, *args, **kwargs):
check_ops(func)
return original(func, *args, **kwargs)
builtins.__build_class__ = build_class
If you look at my actual code it basically injects a wrapper class in place of the module
brisk zenith
i see "DumbassWrapper" and i immediately want a PR
wind maple
ignore debug prints
slim echo
The wrapper class uses __setattr__ and __delattr__ to prevent assignment and deletion
brisk zenith
yeah i noticed, it's a pretty good idea.
oh, and fucking with the __main__ module is what i've been thinking of @wind maple
wind maple
lol
brisk zenith
it's definitely interesting, yeah.
slim echo
I've seen __name__ == '__main__'
brisk zenith
but yeah, your script can import itself if you really want that
slim echo
I don't get syntax highlighting for that and if I type it it expands to py if __name__ == '__main__': pass
brisk zenith
import __main__
x = "hello"
print(__main__.x)```i thought that it just presented a copy of the script as a module but nope, x and __main__.x share the same identity
slim echo
Lol I got AttributeError when I put py import __main__ del __main__.Maybe in my maybe code
I wonder what use you could have for while Maybe
calm rampart
yeah, it's the other way around, if you import the script by name you get a copy
just helped someone with that issue in another channel
sick hound
tropic night
This look nice, I'm in a cabin on the mountain, but I'll try something when I get home
brisk zenith
i'll also apologising in advance, i've got a pretty funky solution coming up
okay, segfaults have been fixed i believe
snow beacon
This ought to be interesting.
brisk zenith
haha :D
stray needleBOT
brisk zenith
i've done something like this challenge before but i implemented it completely differently and it was only half as cool because it only worked in the global scope and segfaulted to high heaven. this new implementation, however, does not. :D
@snow beacon is your code ready for merge? you can make a new PR at any time with any changes you wish to make, but is it done for now? :)
the only issue with my submission is that it doesn't work when imported in the REPL, but that's okay. i'm proud of it anyways :D
import __maybe__
import dis
dis.dis(lambda: Maybe)
# 4 0 LOAD_CONST 1 (<function _rand_bool at 0x7fc239e6e6a8>)
# 2 CALL_FUNCTION 0
# 4 RETURN_VALUE
shhhh :)
mine is actually quite simple in how it works. it just goes through the bytecode and replaces Maybe with _rand_bool() and also raises a SyntaxError if it finds any misuse of Maybe along the way.
crisp pendant
Haha I did that except worse
brisk zenith
oh no, what did you do? :D
brisk zenith
fuck, i see why you apologised :D
crisp pendant
I doesn't work for loops but I've kinda dug myself into a hole so I'm just gonna leave it
brisk zenith
haha, that sounds phenomenal
crisp pendant
Submitting pull rn
brisk zenith
this seems like a challenge that @sick hound would like to do. i think they could make an interesing submission if they're up for it :^)
snow beacon
I'm not done with my submission, but you can merge it now if you want to; I'm trying to make it all more concise using metaclasses, and then I might look at the repl issue. misread, no repl issue
brisk zenith
oh it's fine then, you can keep working on it. just let me know when you think it's ready for merge :D
snow beacon
Will do, thanks.
I've got the metaclass to work, (including with attributes like .numerator which I didn't have before Edit: actually yes I did, I just don't understand how), but it uses dir() which doesn't seem fit for proper code. Is there an equivalent guaranteed to return all of the attributes?
brisk zenith
i mean, you could use __dict__ if possible (or vars(...) which is the same thing)
snow beacon
Hmm. I tried False.__dict__ but it doesn't have one. bool.__dict__ and int.__dict__ return different values (int's are a superset of bool's).
Maybe I'd need to check bool's __dict__ and the __dict__s of all of its superclasses
tropic night
All your solutions are so different from what I have in mind
Maybe I'm just thinking about this the wrong way
And I'm not coding this monstrosity on mobile
brisk zenith
hey that's not a bad thing! the different solutions are the interesting ones and that's exactly what we're looking for. give it a shot when you're able to, i'm looking forward to seeing what you make :D
snow beacon
@brisk zenith What was the issue you encountered importing the module from the repl? I'm on Windows 10, and it works fine for me.
brisk zenith
whose code is this?
brisk zenith
oh haha sorry, i should have clarified. no worries :D
snow beacon
There's a feature I'm having trouble supporting: adding Maybe to itself. It always returns a Maybe rather than converting the Maybes to bools and adding them.
I could convert any Maybes in function arguments to bools beforehand, but it'd not be great for performance to check the type of every argument.
brisk zenith
oh hmm i see what you mean
i guess performance isn't too important though, this isn't production stuff
snow beacon
Yeah. I implemented that, but now I have another problem.
It doesn't work for Maybe and Maybe because of how and is optimised
snow beacon
2 JUMP_IF_FALSE_OR_POP 6
4 LOAD_GLOBAL 0 (dis)
>> 6 RETURN_VALUE```Oh, I was testing it with dis rather than Maybe, but it should be the same
Hmm, I might be wrong as to why it does what it does, but it's giving me wrong results.
Doing a test of 100 Maybe and Maybe is ~50, but with _val() and _val() where _val() is the function I use to generate a random boolean, it's 25.
brisk zenith
what's it doing then?
snow beacon
Hmm.
I think it might be returning a Maybe object rather than a boolean.
The a and b expression returns a if, after converting a to a bool, it ends up being falsey.
Otherwise, it returns b
a and b aren't booleans though
So no matter what, it returns Maybe, which has a 50-50 probability distribution
I don't think this works, but if calling __bool__ were to convert the original Maybe to whatever the result is? That way the and statement would return a proper boolean result. The problem is, it would affect other code. python if Maybe: print(Maybe) would always print True or nothing at all.
To be honest, I don't see any way to get it all to work out without bytecode manipulation, because there's no way to discriminate between different types of call to __bool__.
I will support & and | but not and and or
slim echo
Override the comparison values to interact with the value?
snow beacon
It's a bit outside the scope of my version. I don't want to mess around with monkey patching or bytecode, just nice, simple objects.
Anyway, I'm ready for my branch to be merged. I can't think of anything I can add or fix.
sick hound
@brisk zenith we've done something weird
dense spire
@sick hound using settrace is some straight evil genius shit. bravo 
super fun implementation
radiant dew
Is it possible (ugly and hacky is acceptable) to assign to the same variable object from two different files by just using the assignment operator?
# file1
foo = 1
# file2 runs
print(foo) # >>> 2
# file2
from file1 import foo
foo = 2
print(foo) # >>> 2
it's possible, but probably not the way you're trying to do it
radiant dew
Well obviously not :D
sick hound
# file1
foo = 1
import file2 # executes file2
print(foo) # 2
# file2
import file1
file1.foo = 2``` this might workit's not quite "just using the assignment operator" but it kind of does what you're looking for
just as long as you don't use it in any actual non-esoteric code :P
radiant dew
nah, doesn't do the job for me :(
sick hound
if you're executing file1 directly you might need to import a module called __main__ instead
stray needleBOT
radiant dew
🙈
sick hound
this channel is only for things that you should never use
radiant dew
You mean my solution should be used?
sick hound
?
no
this channel is for python weirdness you shouldn't use, so don't use any of the python weirdness in this channel
radiant dew
I'm aware of that
brisk zenith
@radiant dew what do you mean when you say you couldn't make Maybe refer to the same object?
interesting solution by the way
radiant dew
In my MaybeSyntaxErrorListener I use Maybe in isinstance() and I wan't to move this entire listener to maybelib.py but that would make the listener check for the Maybe object declared in maybelib.py while Maybe = "hello!" in maybe.py would overwrite the imported Maybe and create a local string variable in maybe.py. The listener would not catch the redeclaration since it's listenening to the Maybe object in the maybelib.py file scope.
Thanks, I'm a noob so no cool black magic involved in this solution!
brisk zenith
is your code just intended to work by running maybe.py, or could it be any script which imports the things from maybelib?
radiant dew
Well, at this point it would just work with maybe.py
brisk zenith
i think you can make the listener work for the Maybe object imported into the main running script by listening to __main__.Maybe
radiant dew
I would've prefered importing the things to anywhere
How do I use it?
if not isinstance(__main__.Maybe, MaybeBoolean): NameError: name '__main__' is not defined
slim echo
@sick hound you can del Maybe sorry lol
>>> Maybe = 1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Starwort\Documents\Python\eso\pastebins_maybe.py", line 7, in t
if o in b'Z[ab'and c.co_names[c.co_code[i*2+1]]=='Maybe':raise SyntaxError
SyntaxError: None
>>> del Maybe
>>> Maybe
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'Maybe' is not defined
>>>```You can't... wait... oops
slim echo
Also uncommenting f.f_trace_opcodes=True fixes that but lets me assign
sick hound
Apparently if the trace function throws an exception it won't be a trace function anymore
And... that's just weird...
slim echo
Oh
Rip
>>> while maybe.Maybe: print('lol')
...
lol
lol
lol```
Someone please come up with a use for this loop
nocturne saddle
You can simulate a probability distribution with it
But, that probably doesn't sound like fun to anyone but me
slim echo
How would you simulate a distribution?
nocturne saddle
You could count the number of iterations it runs, do that repeatedly
slim echo
runs = 0
allruns = 0
while True:
runs += 1
total = 0
while maybe.Maybe:
total += 1
allruns += total
print(allruns / runs)
?
nocturne saddle
Depends a bit on what you want
You can also look at the distribution of runs it makes (the number of 0, 1, 2, 3, ....)
slim echo
Average runs ~= 1
nocturne saddle
So, I don't have your code on my PC, but an example of what I mean:
nocturne saddle
!e
from random import randint
from collections import defaultdict
counts = defaultdict(int)
for _ in range(1000):
count = 0
while randint(0, 1):
count += 1
counts[count] += 1
print(counts)
@nocturne saddle Your eval job has completed.
defaultdict(<class 'int'>, {6: 14, 0: 492, 1: 240, 9: 4, 2: 141, 7: 4, 3: 57, 4: 36, 10: 1, 5: 9, 8: 2})
It's not sorted yet, though
radiant dew
@brisk zenith
Oh yeah, but I still have to run the listening thread from __main__?
from maybelib import MaybeSyntaxErrorListener, Maybe, is_instance as isinstance
msel = MaybeSyntaxErrorListener()
msel.start()
# Test Maybe
for _ in range(10):
print(Maybe)
for _ in range(10):
if Maybe:
print("And this code might run if it feels like it")
print(isinstance(Maybe, bool))
Maybe = "hello!"```really? i don't see why that would affect it
radiant dew
If I run it in maybelib it doesn't find any Maybe.
import __main__
from random import choice
from threading import Thread
class MaybeBoolean():
def __repr__(self):
return choice(["True", "False"])
def __bool__(self):
return choice([True, False])
def is_instance(obj1, obj2):
if isinstance(obj1, MaybeBoolean):
return True
return isinstance(obj1, obj2)
class MaybeSyntaxErrorListener(Thread):
def run(self):
while True:
if not isinstance(__main__.Maybe, MaybeBoolean):
raise SyntaxError("can't assign to keyword!")
Maybe = MaybeBoolean()
msel = MaybeSyntaxErrorListener()
msel.start()
if not isinstance(__main__.Maybe, MaybeBoolean): AttributeError: module '__main__' has no attribute 'Maybe'
slim echo
Because __main__ isn't maybelib?
radiant dew
It shouldn't be
brisk zenith
are you still importing Maybe from maybelib in your main script?
radiant dew
Yes
glacial rampart
🐍 ctypes.memset(id(True), 0, ctypes.sizeof(ctypes.c_ssize_t))
2918602020
🐍 1 == 1
True
Invalid address 0xadf65524 passed to free: value not allocated
fish: Job 2, 'command python -q $argv' terminated by signal SIGABRT (Abort)
i don't think that's how you use ctypes.memset
or at least, overwriting True in memory would be significantly more difficult
brisk zenith
integer = 4
memory_copy = duplicate(integer)
normal_copy = integer
print(integer == memory_copy) # true
print(integer is memory_copy) # false
print(integer is normal_copy) # true
decided to get back to work on my memory utils stuff. i've been meaning to make a gist or repo for it, i'll do that soon. this duplicate function does exactly what you'd think, makes an exact copy of an object in memory, even things like small integers which are cached by the interpreter.
sick hound
>>> 42
42
>>> (c_char*28).from_address(id(42))[24] = 43
>>> 42
43
>>> (c_char*28).from_address(id(42))[24] = b'*'
>>> 42
42```that's less elegant though :D
sick hound
>>> int(True)
42```we made True equal to 42
>>> int(True)
0``` we made True equal to 0 but nothing seems to have broken>>> True == False
True``` This is interesting, True is now equal to False>>> int(True)
0
>>> int(False)
1``` We swapped True and False but nothing has changedtf?
this is #esoteric-python, where you can do a few things to change the values of numbers
like?
like above?
yes
from sys import*;from ctypes import*
def a(b):return (c_char*len(b)).from_address(id(b)+32)
def b(c,d,e):
if d=='call':f=c.f_code;g=a(f.co_code);g[:8]=b'd\0F\0d\0S\0'
return b
settrace(b)``` if you run this in a REPL, or with `-i`, the interpreter will print the first constant of whatever code you write instead of actually doing anything...except for when it doesn't work, because apparently it doesn't work sometimes?
oh
if the code is too short it won't work
brisk zenith
oh yeah pastebin, can you please put an ungolfed/commented version of your Maybe submission in the PR? just in a docstring near the actual code will do. it's amazing to see how small you've made it, but i'd also like to see how it works :D
sick hound
we've added an ungolfed version and made the golfed version even smaller
nocturne saddle
It looks really cool @sick hound
Although I'm not really an esoteric python expert (not at all)
sick hound
in a way, it's both the least hacky and the most hacky way that could have been done
sick hound
sys.settrace is meant to be used, it's for implementing debuggers
it sets the trace function
brisk zenith
well yeah i figured that much :D
sick hound
it receives events when a function is called, when a function returns, every line, when an exception is thrown, and in 3.7 optionally every instruction
it takes as arguments the frame object, the event (str), and an argument depending on the event
brisk zenith
oh wow, i never knew that.
sick hound
the return value is what trace function to keep using, or None to stop tracing
in this case it just returns itself, which means it will receive events from every piece of python code, everywhere
brisk zenith
oh i see
sick hound
the frame is a normal frame object, and contains a reference to a code object, which means we can have fun with the bytecode although it can't be changed without ctypes
brisk zenith
ah, so that's why your Maybe is static for each function call
sick hound
the only things that don't get traced are the actual trace function and builtin/C modules
it's not static for every function call, it's static for every line. the trace function gets line events and uses those to update Maybe
brisk zenith
ohh okay that's fair enough then
i wonder, is it possible to overwrite the bytecode of __main__ using ctypes?
i don't know any way of getting at the bytecode in memory of a module without making a copy of it through compile and inspect.get_source
sick hound
frame objects have an f_code that references the code object
f = sys._getframe()
# just get the top frame...
while f.f_back is not None:
f = f.f_back
# f is the top frame, probably __main__
code = f.f_code
# now we have a code object```what if it's not in the __main__ module though?
sick hound
well because of the f_back stuff, it will keep climbing up the stack frame until it gets to the top
brisk zenith
oh okay neat
sick hound
although that won't work if it's from a separate thread
brisk zenith
so, is that the bytecode in memory, or is it a copy?
sick hound
that is the bytecode in memory
brisk zenith
laughs evilly in spanish
sick hound
you get the frame for __main__ executing, get the code object, and get the bytecode from it
if you want to really make sure you're getting the __main__ and not the top level of a separate thread, you could also sys._current_frames(), get the frame from that that goes all the way up to __main__, then use that, but if you're not in a separate thread or async madness or anything like that you don't need to do that
brisk zenith
fair enough
sick hound
anyway, we have to go now, have fun with the bytecode and we'll hopefully see what evil stuff you make with it later
brisk zenith
haha okay, cya. i hope i can do some interesting stuff with the bytecode.
well, this doesn't obliterate the interpreter, so surely it can't be the bytecode. ```py
frame = sys._getframe()
while frame.f_back is not None:
frame = frame.f_back
ctypes.memmove(
id(frame.f_code),
id(None),
type(None).sizeof(None)
)
print("hello")
(i find it amusing how i know it's not right when "it doesn't obliterate the interpreter")
brisk zenith
i've decided i'm gonna make a full module of these fucky things, like __future__ but better. i've made this one work on a per-scope basis, too :D ```py
def test_function():
from black_magic import maybe
for _ in range(3):
print(Maybe) # True, False, False
print(Maybe) # NameError```
fallow cairn
I have got it to do Maybe.maybe
fallow cairn
Can't get it to work
import random
class Maybe:
@property
def maybe(self):
return bool(random.getrandbits(1))
Maybe = Maybe()
print(Maybe)```That is Maybe.maybe
Looks like you would really need to do some major trickery with classes
Or just modify python itself
snow beacon
I opted for the major trickery with classes option
sick hound
@brisk zenith the bytes object is definitely the bytecode
it might be too late to mess with other things though
from ctypes import *;from sys import _getframe;f=_getframe()
while f.f_back is not None:f=f.f_back
(c_char*7).from_address(id(f.f_code.co_code)+114)[:]=b'd\x07F\0d\3S';input("hello world")```this code modifies itself and does not ask for input
it uses the bytecode
brisk zenith
you have a bad habit of golfing code that i want to actually read haha. it's fine, i understand that one. why didn't my original code break though?
sick hound
i'm guessing it's because the code object has already been read and copied somewhere else, so you can't change it
or because it's being treated as a code object and the code just doesn't care that it's actually None
from ctypes import *
from sys import _getframe
f = _getframe()
while f.f_back is not None:
f = f.f_back
(c_char * 7).from_address(id(f.f_code.co_code) + 114)[:] = b'd\x07F\0d\3S'
input("hello world")``` also here's an ungolfed version if you want itbut it's harder than you would think to change because of the magic number 114
brisk zenith
well, i know that bytes objects have memory overhead (so, the content of the bytestring b"hello" would actually be 32 bytes after its id) and then 114 - 32 is 82 which is where the modified instructions begin. i can figure out that much
so it's not really a magic number
if you can figure out which instruction you want to fuck with, it shouldn't be too difficult.
because like, this is an example of the exact data in memory of b"hello there":
b'\x03\x00\x00\x00\x00\x00\x00\x00\xc0M\x18\x95A\x7f\x00\x00\x0b\x00\x00\x00\x00\x00\x00\x00X\xbb\xf7I\x11\xa6h\xfahello there\x00'
just ignore the first 32 bytes and then you can fuck with the content.
sick hound
if you're careful you can also mess with the length of bytes ```py
b = b'abcdefg'
(c_char*32).from_address(id(b))[16] = 255
b
b'abcdefg\x00\xb8\xf4\x88\xe4\xc1\x01\x00\x00H\xe7R\xe4\xc1\x01\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x90\xf4\x88\xe4\xc1\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x08\x00\x00\x00\x01\x00\x00\x00\xff\xff\xff\xff\x90\xf4\x88\xe4\xc1\x01\x00\x00\xe0\xf4\x88\xe4\xc1\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x08\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\xe0\xf4\x88\xe4\xc1\x01\x00\x00h\xf4\x88\xe4\xc1\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x08\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00h\xf4\x88\xe4\xc1\x01\x00\x00\x08\xf5\x88\xe4\xc1\x01\x00\x00\xc8\xe6R\xe4\xc1\x01\x00\x00\x01\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x000\xf5\x88\xe4\xc1\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x19\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x000\xf5\x88\xe4\xc1\x01\x00\x00X\xf5\x88\xe4\xc1\x01\x00'```
brisk zenith
oh okay, so the 16th byte in the "header" of the bytes object dictates its length?
sick hound
well it's probably actually an integer or something, but yes
brisk zenith
yeah i figured.
will that dynamically re-allocate the memory or might it overwrite any objects/data which is directly after the object in memory?
actually what am i saying
yeah, i get it now
sick hound
it just carries on into the data straight after it
sick hound
like a char* missing a null terminator in C
it just keeps going even if it goes into other objects
brisk zenith
mhm
well, here's my main concern. if i'm going to write to a code.co_code in memory, what if my new instructions occupy more space than the original? i obviously wouldn't want to overwrite any important data after the object.. my only idea for this would be to create a new memory buffer elsewhere, write the new bytes data to it, and then replace all in-memory references (pointers) to the old object with pointers to the new object, but that seems wayyyyyyy too crazy an idea.
to be fair, if you could do that then you've basically got the power to do literally anything
sick hound
every code object has at least 4 bytes of code
put something on the stack, and return it.
sick hound
the bytes object also has a null terminator
so you have enough space to write "push constant 0, call with 0 arguments, return"
constant 0 is your own function that you overwrite into co_consts and can be anything you want
sick hound
co_consts always contains at least one value and you can overwrite that value into your function
brisk zenith
yeah, okay.
sick hound
>>> code.co_consts
((((((((((((((((((((((((((((((<NULL>, (<class '_ctypes.Structure'>,), {(((((((<NULL>, 1), 2), 3), 18), 6), None), None): 0}), 'BigEndianStructure', <class '_ctypes.Structure'>), None, None), (<class '_ctypes.PyCFuncPtr'>,), {'code': 6160}), 'CFunctionType', <class '_ctypes.PyCFuncPtr'>), (), 1), 'This class represents a dll exporting functions using the\n Windows stdcall calling convention, and returning HRESULT.\n HRESULT error values are automatically raised as OSError\n exceptions.\n ', None), 'l', None), 'This class represents a dll exporting functions using the\n Windows stdcall calling convention.\n ', None), 'This class represents the Python library itself. It allows\n accessing Python API functions. The GIL is not released, and\n Python exceptions are handled correctly.\n ', None), 'u', None), 'pointer', '_pointer_type_cache'), '?', None), 'P', None), 'c', None), 'b', None), 'B', None), 'Q', None), 'q', None), 'g', None), 'd', None), 'f', None), 'I', None), 'i', None), 'L', None), 'l', None), None, None), None, None), 6160, <class 'ctypes.py_object'>), 2, 3)``` WHOOPSoh haha
scarlet osprey
Excuse me
sick hound
if you ever overwrite an object with another object using ctypes, make sure there is at least one reference to it
fallow cairn
Ok then
sick hound
we set the co_consts to the tuple (1, 2, 3), but it had 0 references
so that happened
@brisk zenith ```py
from ctypes import *
from types import *
from sys import *
def overwrite(code, callable):
if isinstance(code, FunctionType):
code = code.code
array = (c_char*5).from_address(id(code.co_code) + 32)
array[:] = b'd\0\x83\0S' # load constant 0, call function, return
cast(id(code.co_consts), POINTER(py_object))[3] = callable # overwrite the first thing in the tuple with the replacement```
this replaces any code object with any callable thing
...kind of
you can't access the arguments, but other than that it works
brisk zenith
yeah i know, i'm doing something with that idea right now
sick hound
is there a way to increase the number of references an object has?
nevermind, found it
brisk zenith
yeah
sick hound
ok, now it works with arguments py def overwrite(code, callable): def wrapper(*args): callable(*_getframe(1).f_locals['args']) pythonapi.Py_IncRef(py_object(wrapper)) if isinstance(code, FunctionType): code = code.__code__ array = (c_char*5).from_address(id(code.co_code) + 32) array[:] = b'd\0\x83\0S' # load constant 0, call function, return cast(id(code.co_consts), POINTER(py_object))[3] = wrapper # overwrite the first thing in the tuple with a wrapper cast(id(code), POINTER(c_char))[24] = 1 # co_nlocals cast(id(code), POINTER(c_char))[32] = cast(id(code), POINTER(c_char))[32][0] | 4 # make the last argument *args varnames = ('args',) pythonapi.Py_IncRef(py_object(varnames)) cast(id(code), POINTER(py_object))[8] = varnames
and keyword arguments ```py
from ctypes import *
from types import *
from code import *
from sys import *
from sys import _getframe
def overwrite(code, callable):
def wrapper():
locals = _getframe(1).f_locals # get the frame's locals
return callable(locals['args'], **locals['kwargs'])
pythonapi.Py_IncRef(py_object(wrapper))
if isinstance(code, FunctionType):
code = code.code
array = (c_char5).from_address(id(code.co_code) + 32)
array[:] = b'd\0\x83\0S' # load constant 0, call function, return
cast(id(code.co_consts), POINTER(py_object))[3] = wrapper # overwrite the first thing in the tuple with a wrapper
cast(id(code), POINTER(c_char))[24] = 2 # co_nlocals
cast(id(code), POINTER(c_char))[32] = cast(id(code), POINTER(c_char))[32][0] | 12 # tinker with flags for *args and **kwargs
varnames = ('args', 'kwargs')
pythonapi.Py_IncRef(py_object(varnames))
cast(id(code), POINTER(py_object))[8] = varnames```
@brisk zenith
it now works with arguments and keyword arguments, and the thing you replace it with can be any callable object
The stack trace can get a bit weird though py Traceback (most recent call last): File "<stdin>", line 1, in <module> File "overwrite.py", line 24, in lol return None File "overwrite.py", line 10, in wrapper callable(*locals['args'], **locals['kwargs']) File "overwrite.py", line 30, in lol_replacement 1/0 ZeroDivisionError: division by zero
brisk zenith
print("hiya!")
def test_function():
print("beep beep") # this does run
from testing2 import replace_main_bytecode
replace_main_bytecode(test_function)
print("hello!") # this never runs
rather than putting the function calling instruction at the beginning of the code object, it puts it at frame.f_lasti so that when execution resumes in the main script, it actually goes straight into the function
so yeah, dynamic bytecode manipulation of the bytecode of the main script is perfectly possible
import __main__
import ctypes
import dis
import sys
BYTES_HEADER_SIZE = 32
CALL_BYTECODE = b'\x01\x00d\x00\x83\x00S\x00'
# POP_TOP
# LOAD_CONST 0
# CALL_FUNCTION 0
# RETURN_VALUE
def get_main_frame():
frame = sys._getframe()
while frame.f_back is not None:
frame = frame.f_back
return frame
def replace_main_bytecode(new_code):
frame = get_main_frame()
offset = BYTES_HEADER_SIZE + frame.f_lasti
buffer = (ctypes.c_char * len(CALL_BYTECODE))
target_data = buffer.from_address(
id(frame.f_code.co_code) + offset
)
target_data[:] = CALL_BYTECODE
constants = frame.f_code.co_consts
ctypes.cast(id(constants), ctypes.POINTER(ctypes.py_object))[3] = new_code
Spooky
sick hound
🍋
fallow cairn
Result for challange:
Any file you want to use it in, just do do from maybe_lib import Maybe
snow beacon
Have you put it in the repository?
brisk zenith
yeah, make a PR for it in the repo. check pins for the specific challenge location and stuff
grave rover
@brisk zenith make sure to test each submission as to what happens when you do del Maybe
sick hound
This is my solution for adding new keywords to language
It creates a patched import function
and when it used, it constructs an AST from given module and raises syntax warnings whenever a keywords is assigned or deleted
import builtins
import random
class Maybe:
def __bool__(self):
return random.choice((True, False))
def __repr__(self):
return f"Maybe?"
def __str__(self):
return repr(self)
def __instancecheck__(self, other):
return other is bool
Maybe = Maybe()
keyword_patcher(('Maybe',), '__main__')
this is the code for maybe
glacial rampart
you can avoid patching builtins.isinstance by overriding __instancecheck__ on the Maybe class
brisk zenith
@grave rover the exact functionality isn't too important. some people might not know how to implement that sort of del protection, and i don't want that to deter them from making a submission. if they want del to be forbidden, it's up to them. after all, it's just a bit of fun :D
sick hound
@glacial rampart true, i dont know why didnt tought that 😄
sick hound
What is this channel
brisk zenith
did you read the channel topic? :D
#esoteric-python is where weird fuckery goes on. want to make Maybe a thing alongside True and False? what about modifying code in memory so that 2 + 2 == 5? you get the idea. black magic.
brisk zenith
yeah. ```py
ctypes.memmove(id(5), id(4), int.sizeof(4))
print(2 + 2 == 5)
i wonder if !eval does it..
!eval ```py
import ctypes
ctypes.memmove(id(5), id(4), int.sizeof(4))
print(2 + 2 == 5)
@brisk zenith Your eval job has completed.
True
@gilded orchid
fallen heath
Don't ask why, ask why not.
brisk zenith
exactly.
sick hound
@brisk zenith ah. i wasnt able to read the channel topic on mobile, i tried. that's cool
snow beacon
If you press the button with the two people in front of each other (or swipe from the right, I believe), it'll pull up a list of people online, and more importantly at the top the channel description.
sick hound
@brisk zenith
from ctypes import *;from mmap import *
m=mmap(-1,100,prot=PROT_READ|PROT_WRITE|PROT_EXEC);m.write(b'\xB8\x79\x00\x00\x00\xC3')
v=cast(pointer(c_char.from_buffer(m)),CFUNCTYPE(c_char))()
print(v.decode())``` this executes '\xB8\x79\x00\x00\x00\xC3' and prints "y"!eval
from ctypes import *;from mmap import *
m=mmap(-1,100,prot=PROT_READ|PROT_WRITE|PROT_EXEC);m.write(b'\xB8\x79\x00\x00\x00\xC3')
v=cast(pointer(c_char.from_buffer(m)),CFUNCTYPE(c_char))()
print(v.decode())
night quarryBOT
sick hound
oof
grave rover
Slowly learning some bytecode
brisk zenith
nice!
grave rover
Progress: it can turn py def do_thing(arg: int = 2, z: str = 5) -> int: y = 2 return arg**2 into ```py
[LOAD_CONST((2, 5)),
STORE_NAME(
'do_thing',
MAKE_FUNCTION(
BUILD_CONST_KEY_MAP(
LOAD_CONST(('arg', 'z', 'return')),
[LOAD_NAME(int), LOAD_NAME(str), LOAD_NAME(int)]
),
[STORE_FAST(y, LOAD_CONST(2)),
RETURN_VALUE(BINARY_POWER(LOAD_CONST(2), LOAD_FAST(arg)))]
)
),
RETURN_VALUE(LOAD_CONST(None))]
only kwargs are difficult to add since it'd not easy to know if they're there
setting VERBOSE = False gives ```py
[(2, 5),
STORE_NAME(
'do_thing',
MAKE_FUNCTION(
BUILD_CONST_KEY_MAP(
('arg', 'z', 'return'),
[LOAD_NAME(int), LOAD_NAME(str), LOAD_NAME(int)]
),
[STORE_FAST(y, 2), RETURN_VALUE(BINARY_POWER(2, LOAD_FAST(arg)))]
)
),
RETURN_VALUE(None)]
and yes I fixed BINARY_POWER just now to invert the order of arguments
all in under 200 lines of readable code (not including pprint)
brisk zenith
neat :D
grave rover
so I basically wrote a tree view of dis.dis in a way too overcomplicated way but who cares
Understanding this will help me abuse it in the future >:D
brisk zenith
haha yes, that's going to be interesting for sure
you already do some interesting stuff so i'm looking forward to seeing what you're capable of doing with bytecode.
grave rover
>>> from hackery import magic_get_dict
>>> def y():
... pass
...
>>> data = magic_get_dict(y.__code__)
Segmentation fault
```darnit>>> from hackery import fix_function
>>> def y():
... pass
...
>>> fix_function(y, b'd\x01S\x00')
>>> y()
Segmentation fault```patching bytecode: done. Next up: patching constants@brisk zenith I did it
grave rover
brisk zenith
oh nice, i never thought of using replace like that
grave rover
It's pretty neat
Doesn't work on if statements yet, this is due to JUMP_FORWARD jumping outside of the function causing a SytemError unknown opcode
brisk zenith
yeah, fixing jumps after messing with bytecode is a bit of a pain
my Maybe implementation has a full section of code for fixing that if you want to take a look at how i did it.
brisk zenith
yeah. for relative jumps, you would also ignore the changes before the jump instruction
grave rover
For now it just uses py bytecode.replace( POP_TOP() + LOAD_CONST(0) + RETURN_VALUE(), RETURN_VALUE() )
brisk zenith
use dis.hasjrel and dis.hasjabs to check if an instruction has a relative or absolute jump
brisk zenith
hot. i've gotta run, cya.
grave rover
Done
wind maple
we rust now
grave rover
yes
snow beacon
That is wonderful
glacial rampart
Termux ?
grave rover
Yup
grave rover
Almost got refactoring jumps done
I can already distinguish between reljump and absjump
grave rover
this exits with exit code 0, so... 👀
the former gets optimized to LOAD_CONST(1), RETURN_VALUE
brisk zenith
how much does it break if i do ```py
for i in range(10):
j = i
return j
this is taking so long just because of all the new opcodes lmapo
yeah current code segfaults
wind maple
f
grave rover
0 SETUP_LOOP 20 (to 22)
2 LOAD_GLOBAL 0 (0)
4 LOAD_CONST 1 (1)
6 CALL_FUNCTION 1
8 GET_ITER
>> 10 FOR_ITER 2 (to 14)
# MISSING HERE
12 JUMP_ABSOLUTE 10
>> 14 POP_BLOCK
16 RETURN_VALUE
Done
actually, that assert will fail due to co_const locations
grave rover
https://github.com/martmists/BytePatches made my code public
brisk zenith
bytecode modifications just make me think "they did surgery on a python"
grave rover
Managed to get const/name/fast optimization done
grave rover
43 0 SETUP_LOOP 22 (to 24)
2 LOAD_GLOBAL 0 (range)
4 LOAD_CONST 1 (10)
6 CALL_FUNCTION 1
8 GET_ITER
>> 10 FOR_ITER 10 (to 22)
12 STORE_FAST 1 (i)
44 14 LOAD_FAST 1 (i)
16 LOAD_FAST 0 (x)
18 BINARY_ADD
20 JUMP_ABSOLUTE 10
>> 22 POP_BLOCK
>> 24 RETURN_VALUE
```how does this sigsegv@grave rover bytecode expert says it doees a BINARY_ADD, popping two from the stack and pushing one then a jump_absolute to the for_iter which does iter.__next__() pushing that onto the stack there's no store_fast so the stack grows infinitely
wind maple
idk I'm not a bytecode expert
grave rover
hmm...
grave rover
aaaand I somehow broke something else
43 0 SETUP_LOOP 24 (to 26)
2 LOAD_GLOBAL 0 (range)
4 LOAD_CONST 1 (10)
6 CALL_FUNCTION 1
8 GET_ITER
>> 10 FOR_ITER 12 (to 24)
12 STORE_FAST 1 (i)
44 14 LOAD_FAST 1 (i)
16 LOAD_FAST 0 (x)
18 BINARY_ADD
20 STORE_FAST 2 (omitReturnVariableName)
22 JUMP_ABSOLUTE 10
>> 24 POP_BLOCK
>> 26 LOAD_FAST 2 (omitReturnVariableName)
28 RETURN_VALUE
``` why does this fail thoughTIL I need to set co_nlocals to len(co_varnames)
grave rover
How I feel now that I'm so deep into modifying bytecode
brisk zenith
you can go deeper :^)
grave rover
What should I do next tho
grave rover
Btw @brisk zenith python has interesting opcodes for creating classes
brisk zenith
oh interesting, so those will be what makes type work as a constructor i assume?
haha constructor, i sound like i'm from java. you know what i mean.
grave rover
(note: decorator was for testing purposes)
It basically just creates a new scope and assigns it to the name
brisk zenith
sounds about right
grave rover
Odd that __new__ and __init__ calls aren't used
LOAD_BUILD_CLASS seems to set flags for the interpreter
So that when the next CALL_FUNCTION is called, it's interpreted as class creation
I wonder if we can fuck with that, lol
snow beacon
So the interpreter only calls __new__ the next time a function is called after class definition?
calm rampart
@grave rover LOAD_BUILD_CLASS puts __builtins__.__build_class__ on stack
__build_class__ finds the metaclass and calls it
>>> exec('class C: pass\nprint(C)', {'__builtins__': {'print': print, '__build_class__': lambda *a,**k:(a,k)}})
((<function C at 0x00000143374AD378>, 'C'), {})```As I expected
calm rampart
__new__ isn't called until,
well, this is all of course for normal classes
as you know, you create classes by calling them like functions
so that's simply type.__call__
which, itself, takes care of calling __new__, and then, if the return value is of the same type, calling __init__.
class creation flow in __build_class__ is:
Find metaclass in kwargs, else type
Create empty dictionary with metaclass.__prepare__
Place er i'm wrong, as you can see this happens in the function before any of the code you write in the class body__module__ and __qualname__ in dict
Call class body function with dict as locals
(the function is compiled in a special mode which allows a locals dict instead of using all quick locals)
Call metaclass, with name and bases as remaining args, kwargs other than 'metaclass'
return class object
[surrounding code assigns class object to name]
calm rampart
there's not much you can do with it that you can't do with a metaclass instead
the one time i've seriously considered it was as an exercise to be able to make a class whose nested classes are subclasses of itself
but I ended up not following through
grave rover
Anyways, any suggestions for bytecode fuckery I could do (or what stuff could be optimized from a bytecode standpoint)
calm rampart
what are you trying to do
grave rover
Oh fuck I forgot to push earlier today
Note: that version has a lot of bugs I've already fixed
calm rampart
@grave rover if you're messing with class creation, be aware the sequence is slightly different for classes that contain calls to super()
grave rover
Oboi
calm rampart
wait nevermind
i'm wrong, the magic is inside the class body function
but do be aware of class creation with arguments and kwargs (e.g. base classes, metaclass)
grave rover
Sadly classes get evaluated before the decorators so I can't mess with it :(
calm rampart
oh
i assumed you were making a byte code patcher that looks for the decorator within the module byte code
grave rover
What I COULD do though is call the decorator beforehand which modifies __build_class__
No this is just a high-level bytecode parser and restructuring tool
calm rampart
note that __build_class__ can't be local to the module, it has to be in __builtins__, though __builtins__ itself can be a local (to the module, not a function) dict instead of the real builtins module
grave rover
def do_magic():
old_bc = __builtins__.__build_class__
def _build_class(*args, **kwargs):
...
__builtins__.__build_class__ = old_bc # change it back when done
return old_bc(*args, **kwargs)
__builtins__.__build_class__ = _build_class
def decorator(cls):
...
return decorator``` ```py
@do_magic() # changes code before class is instantiated
class X:
pass
I would probably make a local builtins just for safety's sake, in case the class imports something
grave rover
True
Actually
I wouldn't know how to do that properly
Since I can't modify locals in non-current frame
calm rampart
by local i mean to your module
grave rover
Uh, code?
calm rampart
I'd just put this at the very top of your file, then you have your own builtins to fuck around with py __builtins__ = {**(__builtins__ if isinstance(__builtins__, dict) else __builtins__.__dict__)}
(you could just do {**__builtins__.__dict__} but sloppy)
grave rover
ah
grave rover
Yea
calm rampart
you need it in the caller's module
grave rover
Which is why I modify global builtins
calm rampart
you could use the same trick though,
caller_globals = ...
old_builtins = caller_globals['__builtins__']
caller_globals['__builtins__'] = {**(old_builtins if isinstance(old_builtins, dict) else old_builtins.__dict__), '__build_class__': _build_class}```How would I get caller globals
calm rampart
frame fuckery
grave rover
Frame fuckery gives a deepcopy() of the globals/locals
calm rampart
that seems unlikely
calm rampart
maybe for locals
locals dict is fake for function call frames, but globals is always real
grave rover
Ah
calm rampart
normal functions - it's real for class body functions and at the module level (same as globals)
>>> def g():
... return sys._getframe(1).f_globals
...
>>> def f():
... return g() is globals()
...
>>> f()
True
>>> g() is locals()
True```Nice
calm rampart
there's probably stuff you could do for thread-safety tbh
like, get the thread id, make your build_class call the real one if it's in a different thread or if it gets called more than once (i.e. for nested classes)
not sure what happens if your decorator gets called at the same time from multiple threads though
add a mutex maybe
grave rover
Found the undocumented locals modification btw
cc @brisk zenith I remember you looking into this
calm rampart
like i said
it works at module level and probably for class bodies, but not normal function frames
wait, didn't see the LocalsToFast thing
grave rover
It's not documented at all
calm rampart
your test's not effective though unless you do it in a function
grave rover
Sure
calm rampart
since the module level doesn't have fast locals
grave rover
calm rampart
def foo():
add_stuff()
b(a)
if False: a, b = None, None # make sure the locals are real fast locals```Ah
calm rampart
if you don't assign them it'll assume they're globals and use non-fast lookup (also since you had them in the module dict, no way to know which it used)
grave rover
Right
Oh but that's where my bytecode fuckery can help
I'm able to dynamically reassign fasts
calm rampart
what about a bytecode decorator to just de-fastify a function
grave rover
The only problem is that it doesn't work for contextmanagers sadly, but those have their own tricks
calm rampart
replace all fast lookups with regular name lookups like in class bodies
calm rampart
someone could, within the function, actually use locals() like in python 2, and use exec('some assignments', locals=locals()) and have it actually work
drawbacks would be that it's slower mainly
calm rampart
another decorator that might be nice, i haven't checked if you have it, would be to replace name lookups and fast lookups with constants
so it doesn't have to look up, say, print by name every time
grave rover
@register_namespace("a", "b", ...)
def thing():
...
# a and b are not defined
with inject_namespace({"a": 10, "b": 5}):
...
# a and b are set here
# and unset here```I can try adding that functionality
(or you could make a PR)
calm rampart
i've never done any bytecode fuckery, i wouldn't know where to begin
the main case, I think, would be to replace global lookups (and global.attribute lookups, for like math.sqrt) with constants. You'd need a way to exclude mutable global variables
grave rover
Check out some of the stuff I already do, it's pretty simple if you do some manual testing
calm rampart
if you want to get really fancy you could evaluate them if it's in a known list of pure functions and the argument is a constant
grave rover
I think _optimize_access might be most interesting, just look for a name/fast that's never stored to (or only once before access) and you can add it as const
calm rampart
actually, what might be interesting
would be to make a whole constant folding pass
to replace math.sqrt(2) with its result, and then evaluate the wider expression that takes place in, to its result
calm rampart
i might, but this isn't something i see myself spending a lot of time on coding for at the moment... that might change though
grave rover
I added a bunch of debugging tools to make it easy (and a pretty_printer.pprint which supports pretty printing)
calm rampart
(but tbh, if i ever get into serious esoteric python it'd probably be at the AST transformation level, with a custom module loader)
bytecode seems unpleasant to work with
i'll look over your stuff though
do you happen to know though
if there's any way to change the return address of a frame, or if there's a way to resume execution of a frame that's not in the current call stack and isn't an async or generator
i tried to make something that allowed transparent conversion of a non-async function to async via a callback, and that's where i hit a wall
calm rampart
i was gonna have the callback throw an exception, and then resume the caller later when the callback's future has a result
grave rover
Can I have pseudocode
calm rampart
ok basically my end goal is this... ```py
def target_function(callback):
...
something = callback()
...
async def fuckery(target_function, async_callback):
def sync_callback():
future = callback()
raise FuckeryException(getframe(1), future)
try:
call target_function(sync_callback)
except FuckeryException:
await future
resume frame after raise
async def caller(...)
await fuckery(target_function, callback)```
grave rover
I see
calm rampart
i need to go, back in 30 minutes or so
grave rover
You could just do raise FuckeryException(future) and use except FuckeryException as exc and exc[0] would be the future. Resuming the frame might be difficult though
calm rampart
yeah it's resuming that's the part i got stuck on
snow beacon
I honestly don't know if this link https://stackoverflow.com/a/46434344 helps with that, because I don't understand what's happening here. Assuming it doesn't, doesn't pdb allow you to execute commands in the context of certain stack frames? (The "!" command.)
calm rampart
like, what i want to do is basically dive back into the "abandoned" (by the exception throw) stack frame, with its same call stack (which is fine because it was called initially from the fuckery function)
snow beacon
https://github.com/ajalt/fuckitpy says it uses "live call stack modification"
I did a ctrl-f for "stack" in the source code and found inspect.stack()[1][0].f_locals[victim] = module
calm rampart
nah that's not going to work
that's for being called from a module level
and at that point it's just assigning the modified module into the caller's namespace
snow beacon
I did a bit of digging, and it looks like they use the AST to surround everything with try/except clauses.
I guess the equivalent would be using something like settrace() and put all your exception logic in the stack frame itself with the code that raises it
You don't have to resume the stack frame yourself that way.
calm rampart
eh
the problem is
i have to be able to suspend the whole stack of the callback (what if the target function doesn't call it directly), and come back to the async caller's frame, and be able to resume the callback or its caller later
a convert_sync_to_async bytecode hack might be easier
snow beacon
Maybe coroutines?
calm rampart
wait isn't that basically what gevent is
the whole exercise is to be able to have a non-coroutine, called by a coroutine, call another coroutine as a callback. Solving it with coroutines is ignoring the problem
snow beacon
Hmm. How easy is it to edit the bytecode interpreter itself?
You would just need to find the pointer to the current stack frame.
And change it of course
calm rampart
no, i figured out it's impossible
the problem is, despite coroutines existing somewhat independently,
snow beacon
That's what the internet seems to think
calm rampart
the call stack isn't really able to be thrown around like I'm trying to do
because frame objects are only half the picture, there could be any intervening amount of C code between one pure-python function and another
snow beacon
Not in its current state, but you could just edit the virtual machine
calm rampart
no because the virtual machine isn't real
you can't do anything about C function calls interleaved with it
snow beacon
Can't you call arbitrary C functions with python code?
calm rampart
yeah
snow beacon
I thought that was the point of the Ctypes module
calm rampart
and those C functions can call other Python functions
right
the problem is the C stack
the thing i'm trying to do can't be done because C functions don't have "frame objects" that can be passed around as almost-first-class values
snow beacon
At some point the problem is possible, but belongs in an #esoteric-c channel.
calm rampart
the best i could do is run the target function in an executor, and have the sync callback schedule the async callback on the main loop and poll its result
which isn't even interesting enough to bother
there's no way to properly suspend and resume the execution of a function that's not a coroutine (generator or async)
snow beacon
I'm trying to learn Lisp, and one of the things I hear they do is "first design the language you want to solve the problem in, then solve the problem in it".
Now, that might be easy enough in FORTH or Lisp, but it's still not impossible in Python, I don't think.
calm rampart
C#
proper async requires language support, winapi uses callbacks and wait loops like everyone else
python's async/await syntax was actually pretty much directly 'stolen' from C#
so it works the same in both languages
arctic bridge
Interesting
sick hound
def trace(frame, event, arg):
if event == 'line':
if frame.f_code.co_code[frame.f_lasti::2].startswith(b'td\x83\1'):
goto, const = frame.f_code.co_code[frame.f_lasti+1::2][:2]
if frame.f_code.co_names[goto] == 'goto':
const = frame.f_code.co_consts[const]
frame.f_lineno += const
return trace``` If you `sys.settrace` that, it implements `goto`use goto as a function on its own at the beginning of a line, with an argument of a constant number to add to the line number
def test():
print('0')
goto(4)
print('2')
print('3')
goto(3)
print('1')
goto(-4)
print('4')```
grave rover
I should implement compiler.py$47 to find the required stack size
brisk zenith
okay so me and pastebean discussed something and we're a bit stumped (and i've never come across something they can't figure out, honestly :D). the frame object in python's internals is defined as a C struct. somewhere in this struct is a code object which then has a co_code attribute (containing the currently-executing bytecode of the frame itself). obviously bytes objects are immutable, but modifying their content in memory is possible. this means that if i modify the memory content of a frame.f_code.co_code, the new bytecode will be executed in that frame if done properly. that's easy enough, and i've demonstrated that before in this channel.
however, this isn't what's confusing me. i've tried to overwrite the memory of the f_lasti of a frame object, but it didn't seem to do anything. i would expect it to change where the frame's execution resumes. if i set it to 32, i would expect the frame to start executing the bytecode at index 32, but instead it just resumes where it left off as if nothing changed. however, for some reason it does work when modified from within a trace function as pastebean shows above. the frame.f_lasti object is stored in the frame struct as a C int, so changing its contents in memory is as easy as replacing the 4 bytes that represent it. here's how i do it: ```py
import sys
from ctypes import *
def goto(offset):
frame = sys._getframe().f_back
lasti = c_int.from_address(id(frame) + 104) # the `frame.f_lasti` is a C int located precisely 104 bytes after the start of the struct
lasti.value = offset - 2 # this is overwriting the memory of `frame.f_lasti`
def testing():
print("first")
goto(22)
print("second") # why isn't this line skipped?
print("third")
it's just very confusing because it works when a frame is modified in a trace function from sys.settrace but not when modified from sys._getframe, despite the frames both having the exact same identity. i've tested it and have confirmed that the frame given in a trace function is the same frame given using sys._getframe:
import sys
def trace(frame, *_):
print(sys._getframe().f_back is frame) # prints True
return trace
sys.settrace(trace)
so, finally, my question is as follows: why does it work when modified from a trace function with sys.settrace, but not when done to a frame from sys._getframe, and how might i get it to work without using sys.settrace?
snow beacon
Question: if you get the frame from settrace, but ignore it and get a frame from _getframe, does altering the _getframe one do what you want to? I would expect they're the same because of the is check, but then again, what you say seems to disprove that. (My guess is that settrace alters the interpreter state so that it allows you to edit the _getframe frame properly.)
If that doesn't work, you could try editing the pointer on the frame that the current one links to, rather than the current one itself—maybe the interpreter caches a pointer to the previous stack frame rather than reading it off the frame.
brisk zenith
oh hold on, i can't get it to work in sys.settrace at all now.. 🤔
am i right in thinking that this should theoretically work? ```py
import sys
from ctypes import *
def goto(offset): pass
def testing():
print("first")
goto(22)
print("second")
print("third")
def trace(frame, event, arg):
if event == "call" and frame.f_code.co_name == "goto":
lasti = c_int.from_address(id(frame.f_back) + 104)
lasti.value = frame.f_locals["offset"] - 2
return trace
sys.settrace(trace)
testing()
however, when checking the sys._getframe().f_lasti after the goto call, it just prints out the normal value.
@sick hound i need your brain
snow beacon
Won't the frame have changed by that time?
Eh, I don't know, I'm going to slink off and golf something that I actually understand.
brisk zenith
def trace(frame, event, arg):
if event == "call" and frame.f_code.co_name == "goto":
print(frame.f_back.f_code is testing.__code__) # prints true when in the code above
return trace
i guess it might have something to do with how the code object is generally unchanging but the f_lasti attribute changes a lot, maybe it caches those.
grave rover
Can this goto impl only jump within the same function?
sick hound
@brisk zenith the way we did it was setting frame.f_lineno, with =
which, after going through a lot of checks to make sure it's not jumping onto an except: or into the middle of a block or anything like that, sets the f_lasti and f_lineno
i'm not really sure why that's different, but it is
grave rover
@brisk zenith @sick hound see if you can port this to py3
Some old goto code I had
I'll pastebin code
sick hound
class MissingLabelException(Exception):
def __getattribute__(self, _):
raise self```Idek
sick hound
in our new implementation, goto and comefrom are both set to a MissingLabelException :P
@grave rover this is more of another implementation that happens to be in python 3 than a port, but we did this https://pastebin.com/56WW1yCR
grave rover
Hm