#process_vm_readv magic. Potentially NOT golang related

32 messages · Page 1 of 1 (latest)

fast wadi
#

Hello! I've ran into a weird issue so far that I have not been able to pin down thus far. Keep in mind this MAY NOT be a golang issue/bug, rather a linux kernel oddity that I haven't picked up on yet.

I'm experimenting with VM_PROCESS_READV to read the heap of a remote process on linux.

the relevant (not complete) code snippet is below, testing with using a local process at the moment. I have no idea what is going wrong. I allocate a local 400 byte buffer, check it again right before the syscall, use the syscall, (which returns it read 400 bytes), check it after the syscall, and the buffer is magically only 100 bytes. Any clue as to why this might be happening?

package main

import (
    "bytes"
    "fmt"
    "os"
    "runtime/debug"
    "strconv"
    "syscall"
    "unsafe"
)

const process_vm_readv = 310

type IOVec struct {
    iov_base unsafe.Pointer
    iov_len  uintptr
}

func main() {
    var pid uint64 = uint64(os.Getpid())
    local := new(IOVec)
    debug.SetGCPercent(-1)
    buf := make([]byte, 400, 400)
    findme := bytes.Repeat([]byte("ABCD"), 25)
    metoo := bytes.Repeat([]byte("YEAH"), 1)
    fmt.Printf("find: %p \n", &findme)
    fmt.Printf("metoo: %p \n", &metoo)
    local.iov_base = unsafe.Pointer(&buf)
    local.iov_len = 400
    remote := new(IOVec)
    remote.iov_base = unsafe.Pointer(&findme)
    remote.iov_len = 400
    fmt.Printf("After syscall: len(buf) = %d, cap(buf) = %d\n", len(buf), cap(buf))
    a, b, c := syscall.Syscall6(process_vm_readv, uintptr(pid), uintptr(unsafe.Pointer(local)), 1, uintptr(unsafe.Pointer(remote)), 1, 0)
    fmt.Printf("After syscall: len(buf) = %d, cap(buf) = %d\n", len(buf), cap(buf))
    fmt.Printf("%v, %v, %v\n", a, b, c)
    fmt.Printf("%#v, %#v, %#v\n", local, remote, buf)

}
desert surge
#

is local.iov_base supposed to be the base pointer that is written to? because *buf is a pointer to the slice, not a pointer to the underlying array, use unsafe.SliceData for the latter

#

more expliciltly, slices in go are currently layed out as go type slice struct { ptr unsafe.Pointer len, cap int }
the expression unsafe.Pointer(&buf) gives you a pointer to this structure, not the inner ptr field

fast wadi
#

ahhhhh

#

that makes sense as to why it doesn't work now

#

but why did it work for 100 bytes?

#

as I feel like it would just be overwriting ptr and would end up crying/crashing

desert surge
#

in your post you did not claim it worked for 100 bytes, you claimed the len field of the slice was 100, that would be perfectly possible given you are ostensibly writing to the wrong pointer
did I misinterpret?

#

the reason why I'm being wishy washy with my assertions is that I do not know what iovec is and how it's supposed to work
if it expects a pointer to which it will write the data then you are definitely giving it the wrong one

fast wadi
#

if I read 100 bytes, and copy it into buf, it works.
if I read 400 bytes, and copy it into buf, it only returns 100.
now with unsafe.SliceData if I read 400 bytes, 400 bytes do get returned.

desert surge
#

what is your criteria to distinguish working from not working? just the length?

#

or are you asserting that the buffer contents are correct?

fast wadi
#

just based off of length at the moment. my goal is to ultimately spray the whole heap to find data within.

desert surge
#

so you cannot distinguish a read through a mapped, but otherwise invalid, pointer from a read through the correct pointer
that is, if you're just looking at the length you cannot make any assertions about whether you trampled the array pointer or not, unless it happens to be trampled to a value that faults

#

if our understanding is correct, you can revert the unsafe.SliceData change and observe the address of buf.ptr before and after the read

#

(to establish if indeed it has always been wrong, even in the 100 case)

#

because up to this point, you still have not confirmed if my assumption that local.iov_base is supposed to be a pointer to the data is correct or not 😁

#

from where I stand, it could just as well accept a pointer to a struct { void* ptr; len usize; }

fast wadi
#

so I don't think so but I'm not 100% sure.

desert surge
#

my assumption was correct, iov_base is supposed to be a pointer to the underlying array and iov_len the len of the slice (given it's a []byte)

#

well, at most len (or cap), I suppose the concrete value is up to you

#

actually no, that definition is not sufficient

fast wadi
#

I think I incorrectly assumed the fix was "right" just because the length matched, not the contents

#

oh no it was right

desert surge
#

the fix being unsafe.SliceData? it is, the syscall wants an array of spans, you are giving it one now

#

here's an abbreviated version: ```go
package main

import (
"bytes"
"fmt"
"os"
"runtime"
"syscall"
"unsafe"
)

const process_vm_readv = 310

type IOVec struct {
iov_base unsafe.Pointer
iov_len uintptr
}

func rawptr[T any](p *T) uintptr { return uintptr(unsafe.Pointer(p)) }

func main() {
findme := bytes.Repeat([]byte("ABCD"), 25)
defer runtime.KeepAlive(findme)

buf := make([]byte, len(findme))
local := []IOVec{{
    iov_base: unsafe.Pointer(unsafe.SliceData(buf)),
    iov_len:  uintptr(len(findme)),
}}
remote := []IOVec{{
    iov_base: unsafe.Pointer(unsafe.SliceData(findme)),
    iov_len:  uintptr(len(findme)),
}}
fmt.Printf("bef: %q, %q\n", findme, buf)
syscall.Syscall6(process_vm_readv, uintptr(os.Getpid()), rawptr(unsafe.SliceData(local)), uintptr(len(local)), rawptr(unsafe.SliceData(remote)), uintptr(len(remote)), 0)
fmt.Printf("aft: %q, %q\n", findme, buf)

}

which yields: ```
bef: "ABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCD", "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
aft: "ABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCD", "ABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCD"```

edited to make the parallel between what `process_vm_readv` wants and what that generalizes to (of course, given you just want 1 element for each, just giving it the pointer and len of 1, as you were doing, is perfectly sufficient)
fast wadi
#

This makes more sense now. time to spray the whole heap and find my data!! thank you for the help, I was stuck on that for way too long...

desert surge
#

@fast wadi I neglected one aspect of my example, the rawptr helper violates the uintptr <=> unsafe.Pointer rules, they should be done inline in the Syscall6 call
I just got off work and it hit me 😅

#

there's more to be aware of, for instance, the iov_base ptr violates the pointer passing rules listed in cmd/cgo https://pkg.go.dev/cmd/cgo
(should be pinned) actually I don't know that this is true, ignore this last statement

#

ok, I got confused with uinptrescapes and uintptrkeepalive, from HACKING.md:

The //go:uintptrkeepalive directive must be followed by a function declaration.

It specifies that the function's uintptr arguments may be pointer values that
have been converted to uintptr and must be kept alive for the duration of the
call, even though from the types alone it would appear that the object is no
longer needed during the call.

This directive is similar to //go:uintptrescapes, but it does not force
arguments to escape. Since stack growth does not understand these arguments,
this directive must be used with //go:nosplit (in the marked function and all
transitive calls) to prevent stack growth.

The conversion from pointer to uintptr must appear in the argument list of any
call to this function. This directive is used for some low-level system call
implementations.
so my last statement regarding the inner pointer of IOVec was indeed 100% incorrect
it also gives context as to why my rawptr helper is illegal, it seems I was only remembering half the story