Fixing UB in random crates | Rust Programming Language Community | Page 1

manic tangle Jul 22, 2022, 4:48 PM

#

is there a way to print under miri without synchronization (not println!)?

tough leaf Jul 22, 2022, 4:49 PM

#

manic tangle is there a way to print under miri without synchronization (not println!)?

does it need to avoid sync ever, or just where you want to print?

could you write to a thread local String?

#

or a static mut [u8; N]

manic tangle Jul 22, 2022, 4:50 PM

#

it's multi-threaded

#

I want to know the order.... okay static mut should work

tough leaf Jul 22, 2022, 4:51 PM

#

do you need synchronization between the printlns then

#

hmmmm

manic tangle Jul 22, 2022, 4:51 PM

#

miri is single threaded right

#

so static mut should work

tough leaf Jul 22, 2022, 4:51 PM

#

it is, yeah

#

it does pre-empt threads occasionaly

manic tangle Jul 22, 2022, 4:52 PM

#

yeah I'm trying to debug a data race

#

need to know more details and println! is changing things

haughty mica Jul 22, 2022, 4:55 PM

#

Please complain here that debugging data races sucks https://github.com/rust-lang/miri/issues/2205

GitHub

Print out both accesses involved in a data race · Issue #2205 · rus...

Currently Miri throws a UB on the access that ultimately causes a data race, which is always the later one. It is not clear where the other access involved in the data race is in the source file. e...

#

It will encourage me to work on it

manic tangle Jul 22, 2022, 5:03 PM

#

static mut does not work

#

miri is detecting a data race ferrisWeary

tough leaf Jul 22, 2022, 5:05 PM

#

manic tangle miri is detecting a data race <:ferrisWeary:797313596065185792>

can you disable the data race detector

manic tangle Jul 22, 2022, 5:05 PM

#

no I'm using it lol

#

okay this works:

const EMPTY: String = String::new();
static mut PRINT_I: AtomicUsize = AtomicUsize::new(0);
static mut PRINTS: [String; 100] = [EMPTY; 100];

pub fn println(x: String) {
    unsafe {
        let i = PRINT_I.fetch_add(1, Ordering::Relaxed);
        PRINTS[i] = x;
    }
}

pub fn flush() {
    unsafe {
        for x in PRINTS.iter() {
            println!("{}", x);
        }
    }
}

tough leaf Jul 22, 2022, 5:05 PM

#

oh no

#

that works

manic tangle Jul 22, 2022, 5:05 PM

#

yes

#

I mean, it's sound

#

actually, I need to call flush right before the data race happens ferrisThink

#

there's no way to tell miri to keep running after a race is there

#

wait, can I open stdout as a File ferrisThonk

#

pub fn println(x: String) {
    unsafe {
        let mut f = std::fs::File::from_raw_fd(1);
        f.write_all(x.as_bytes()).unwrap();
        std::mem::forget(f);
    }
}

#

ferrisBut

#

it works

tender nimbus Jul 22, 2022, 5:54 PM

#

manic tangle miri is detecting a data race <:ferrisWeary:797313596065185792>

data racing to detect the data race

tender nimbus Jul 22, 2022, 8:13 PM

#

https://twitter.com/sunshowers6/status/1550545015667011584 that's pretty cool, running each test in isolation

rain 🌧 (@sunshowers6)

Super excited to announce that nextest now works with the Miri interpreter for @rustlang! Simply install the latest version of cargo-nextest, and Miri from master (nightly coming soon). Then, run:

cargo miri nextest run

https://t.co/vNEoD5Abom

tough leaf Jul 22, 2022, 8:14 PM

#

tender nimbus https://twitter.com/sunshowers6/status/1550545015667011584 that's pretty cool, r...

thankfully they clarified that they weren't about to spin up THREAD_COUNT miri instances at once :P

tender nimbus Jul 22, 2022, 8:15 PM

#

ferrisWhen

tough leaf Jul 22, 2022, 8:15 PM

#

which would oom so fast you wouldn't believe

proper belfry Jul 22, 2022, 8:21 PM

#

cargo-nextest is something my instinctual reaction of “just contribute to cargo” has forbade me from trying so far

#

but maybe I should

tender nimbus Jul 22, 2022, 8:22 PM

#

There are many big very hard challenges around interesting this into cargo, that's why it is its own thing

proper belfry Jul 22, 2022, 8:30 PM

#

tender nimbus There are many big very hard challenges around interesting this into cargo, that...

Out of curiosity, what are they? Is there anything stopping them just having cargo test-next with an entirely different codebase than cargo test?

tender nimbus Jul 22, 2022, 8:31 PM

#

I don't remember specifics

thorny venture Jul 22, 2022, 8:54 PM

#

proper belfry cargo-nextest is something my instinctual reaction of “just contribute to cargo”...

Cargo nextest is really nice, use it. However it doesn't run doctests

proper belfry Jul 22, 2022, 8:55 PM

#

alright, I’ll stomach it :P

thorny venture Jul 22, 2022, 8:55 PM

#

And I think it is intended to move fast for now rather than be stuck by cargo

#

Eg cargo has unstable apis for ages now, and I can't use them in CI properly. Nextest just worked to get my nice junit output

proper belfry Jul 22, 2022, 8:57 PM

#

also conrad you may be interested in the discussion starting about here <#rust-discussions-3 message>

proper belfry Jul 22, 2022, 8:58 PM

#

thorny venture Eg cargo has unstable apis for ages now, and I can't use them in CI properly. Ne...

fair point

#

it seems like we need better processes for development of tools like Cargo so experimentation can usefully happen in-tree

thorny venture Jul 22, 2022, 9:01 PM

#

I think nextest also doesn't guarantee that your tests will work. If you have some sync in your tests it might fail because nextest is more parallel by default. Our test suite at work required fixing to use nextest

#

In fact, I had to debug annoying parallel test failures today. -j 1 all tests passed. In parallel a whole module of tests failed :(

#

The solution ended up being making a separate database table for each test ferrisClueless

#

For complicated reasons that I can't really explain but it was my whole day debugging why sometimes the test passed and sometimes it failed

manic tangle Jul 22, 2022, 9:04 PM

#

sounds fun

thorny venture Jul 22, 2022, 9:06 PM

#

Essentially we run background workers that process jobs in batches, we made a change that intentionally failed the batch job (which gets picked up later), but since its synced in the DB, some other tests were processing it in a different test context, so the actual test didn't detect the work being done

#

So we isolated the DBs so only that one test could process the jobs it scheduled

haughty mica Jul 22, 2022, 10:49 PM

#

The only difference is that nextest runs different test targets in parallel

#

If your tests can't tolerate running in parallel that's mad sketch

tough leaf Jul 22, 2022, 10:50 PM

#

test runner that runs tests concurrently with themselves to try and find tests that can't handle that

#

as in, running the same test twice

#

in parallel :P

haughty mica Jul 22, 2022, 10:50 PM

#

cargo test already runs tests within a target in parallel, in the same process, which is super surprising to some people

tough leaf Jul 22, 2022, 10:51 PM

#

i suppose the solution nowadays is a static mutex of () for each Thing that can't handle being run in parallel

#

or better yet fix your tests so you operate on isolated things

haughty mica Jul 22, 2022, 10:53 PM

#

I think nextest breaks the static mutex trick

#

You can't synchronize across test targets with it

west violet Jul 23, 2022, 12:12 AM

#

Shmem locks

tender nimbus Jul 23, 2022, 6:37 AM

#

tough leaf i suppose the solution nowadays is a static mutex of () for each Thing that can'...

nextest runs each test in a separate process

haughty mica Jul 23, 2022, 6:55 AM

#

And it's glorious

ebon orbit Jul 23, 2022, 1:02 PM

#

Henlo

#

@haughty mica https://miri.saethlin.dev/ub?crate=ferris-says&version=0.2.1 is old

ferris_says already updated their smallvec version 9 days ago

#

sighhh is miri not available on latest nightly again

haughty mica Jul 23, 2022, 1:09 PM

#

Hm

#

I'll see what I can do

#

Oh, there's no new release

ebon orbit Jul 23, 2022, 1:11 PM

#

oh lol

#

what was that link for component availability again?

#

https://rust-lang.github.io/rustup-components-history/

nvm found it

haughty mica Jul 23, 2022, 1:11 PM

#

Also I don't use that, just FYI

ebon orbit Jul 23, 2022, 1:12 PM

#

haughty mica Also I don't use that, just FYI

wdym?

haughty mica Jul 23, 2022, 1:12 PM

#

https://github.com/saethlin/miri-tools/blob/02e325a1454693f92d9512c32217d7027a0edb5b/docker/Dockerfile#L7

GitHub

miri-tools/Dockerfile at 02e325a1454693f92d9512c32217d7027a0edb5b ·...

Tools for finding experimental UB. Contribute to saethlin/miri-tools development by creating an account on GitHub.

ebon orbit Jul 23, 2022, 1:13 PM

#

ah

#

help my rustup folder is 36 gigabytes

#

oo

#

ferris_says no longer has UB

haughty mica Jul 23, 2022, 1:17 PM

#

It's also possible I just need to rerun it

ebon orbit Jul 23, 2022, 1:17 PM

#

haughty mica I'll see what I can do

smallvec is also not tested on latest release, miri also can't seem to find UB on latest release of it

haughty mica Jul 23, 2022, 1:18 PM

#

The latest smallvec release is not on the ub page

#

https://miri.saethlin.dev/?crate=smallvec&version=1.9.0

ebon orbit Jul 23, 2022, 1:19 PM

#

oh

#

mbmb

ebon orbit Jul 23, 2022, 1:20 PM

#

haughty mica The latest smallvec release is not on the ub page

Right but what's exactly the point of still having an older release on the UB page ferrisThonk

haughty mica Jul 23, 2022, 1:20 PM

#

But it isn't there

#

Oh

#

Link rot

ebon orbit Jul 23, 2022, 1:21 PM

#

haughty mica But it isn't there

haughty mica Jul 23, 2022, 1:21 PM

#

Smh

#

Oh I know what's up

ebon orbit Jul 23, 2022, 1:23 PM

#

i think i might be able to send a pr to update smallvec in rhai to prevent UB

haughty mica Jul 23, 2022, 1:23 PM

#

The code that renders that page is broken, I broke it a few days ago

ebon orbit Jul 23, 2022, 1:25 PM

#

groan tests in miri are so slow

tender nimbus Jul 23, 2022, 1:26 PM

#

they are a lot faster than they once were

haughty mica Jul 23, 2022, 1:44 PM

#

If your Miri is relatively recent

#

Sounds like gimbles has a bazillion toolchains

haughty mica Jul 23, 2022, 4:19 PM

#

Ok site should be repaired now

#

Missing spans melobonk https://miri.saethlin.dev/ub?crate=jpeg-decoder&version=0.2.6

ebon orbit Jul 23, 2022, 6:55 PM

#

haughty mica Sounds like gimbles has a bazillion toolchains

Not that many

tender nimbus Jul 23, 2022, 8:09 PM

#

@haughty mica a comment on the StackCache reads:

/// This maps items to locations in the borrow stack. Any use of this still needs to do a
/// probably-cold random access into the borrow stack to figure out what Permission an
/// SbTag grants. We could avoid this by also storing the Permission in the cache, but
/// most lookups into the cache are immediately followed by access of the full borrow stack anyway.

isn't this incorrect, now that Item is bit packed and therefore the whole item is stored in the stack?

haughty mica Jul 23, 2022, 8:20 PM

#

Yes that is incorrect

#

The whole item is stored in the cache

tender nimbus Jul 23, 2022, 8:29 PM

#

https://github.com/rust-lang/miri/pull/2427

haughty mica Jul 23, 2022, 8:37 PM

#

This is my second delegate notlikeblob

tender nimbus Jul 23, 2022, 8:39 PM

#

ferrisBut

haughty mica Jul 23, 2022, 8:43 PM

#

At work we're trying to make a short list of things to tell people not to do here, as sort of like ground rules. The most-agreed-upon one is

Don't rubber-stamp a PR just because Ben approved it

#

Btw I am trying to review this but English is hard

haughty mica Jul 24, 2022, 11:37 PM

#

The nextest maintainer is a machine

tough leaf Jul 24, 2022, 11:37 PM

#

also
do you know anything about the -Zmiri-retag-fields flag

haughty mica Jul 24, 2022, 11:38 PM

#

Yes

tender nimbus Jul 24, 2022, 11:38 PM

#

haughty mica The nextest maintainer is a machine

do they also fork themselves to run in parallel?

tough leaf Jul 24, 2022, 11:38 PM

#

should i be going and investigating the cases where the stdlib fails with it

#

or is that a known issue

haughty mica Jul 24, 2022, 11:38 PM

#

Does it?

tough leaf Jul 24, 2022, 11:38 PM

#

yes

tender nimbus Jul 24, 2022, 11:38 PM

#

tough leaf also do you know anything about the `-Zmiri-retag-fields` flag

it... retags fields on non-box types ferrisBut

haughty mica Jul 24, 2022, 11:38 PM

#

Please report them

tough leaf Jul 24, 2022, 11:39 PM

#

📎 message.rs

haughty mica Jul 24, 2022, 11:39 PM

#

I think I already ran the stdlib tests with it

tough leaf Jul 24, 2022, 11:39 PM

#

yeah i'm using miri-test-libstd

#

i forget if core passes or if i just got bored

haughty mica Jul 24, 2022, 11:40 PM

#

Oh fuck yeah

tough leaf Jul 24, 2022, 11:40 PM

#

alloc is where i expect the more spicy stuff to be in any case

haughty mica Jul 24, 2022, 11:40 PM

#

I'm so happpy

#

Let me see if I can repro this

tough leaf Jul 24, 2022, 11:40 PM

#

command's at the top btw

#

exact test

haughty mica Jul 24, 2022, 11:40 PM

#

Yeah

tough leaf Jul 24, 2022, 11:41 PM

#

i'll let core tests run

#

see if anything fun is found there

#

either a failure or an OOM

#

take your pick

haughty mica Jul 24, 2022, 11:43 PM

#

Awesome

#

You should report this

tough leaf Jul 24, 2022, 11:43 PM

#

where do miri failures go, rust-lang/rust ?

haughty mica Jul 24, 2022, 11:43 PM

#

I can't figure out how to prevent my vec-box branch from running into this so I though it was just my fault because I could have sworn the whole thing passed cleanly before

#

Yes probably

#

The flag works as intended I think

tough leaf Jul 24, 2022, 11:44 PM

#

ralf Has Awakened (i got 2 notifs from him replying to things i'm in, trying to push them forward)

haughty mica Jul 24, 2022, 11:44 PM

#

I can write this up and credit you if that's what you're asking

tough leaf Jul 24, 2022, 11:44 PM

#

nah i'll do it

haughty mica Jul 24, 2022, 11:51 PM

#

Now this is podracing

#

With my unix-exec branch, nextest now works perfectly with Miri

tough leaf Jul 25, 2022, 12:03 AM

#

okay reported
https://github.com/rust-lang/rust/issues/99701

GitHub

Miri `-Zmiri-retag-fields` libstd test failure inside `VecDeque::dr...

Using https://github.com/rust-lang/miri-test-libstd, running MIRIFLAGS="-Zmiri-retag-fields" ./run-test.sh alloc --all-targets collections::vec_deque::tests::test_drain, I get run...

#

i didn't ping ralf ~~because i didn't want to annoy him even though i don't think he'd mind~~ because i just know he'll find it anyways

haughty mica Jul 25, 2022, 1:55 AM

#

tough leaf okay reported https://github.com/rust-lang/rust/issues/99701

Holy shit is this a retag of the base tag as SRO???

#

That would be omegaroll

#

Ah yes, you have passed this VecDeque to a function, IT IS NOW IMMUTABLE

#

Oh wait

#

No this is so confusing

#

We're trying to use the base tag for writes and we can't because we got a SRO retag from a function argument

tough leaf Jul 25, 2022, 1:57 AM

#

aren't we passing ownership of the Drain

#

oh

#

hm
drain internally is using an Iter (immutable reference)

haughty mica Jul 25, 2022, 1:59 AM

#

Oh

#

Yeah

#

Oofies

tough leaf Jul 25, 2022, 1:59 AM

#

is that where the SRO comes from

#

i assume so

haughty mica Jul 25, 2022, 1:59 AM

#

probably

#

field retagging is rough because the span is just like "this line" and it could have recursed through who knows how much type

tough leaf Jul 25, 2022, 2:00 AM

#

haughty mica Jul 25, 2022, 2:00 AM

#

l m a o

tough leaf Jul 25, 2022, 2:00 AM

#

does

does using iter invalidate deque

haughty mica Jul 25, 2022, 2:00 AM

#

There's no invalidation here

tough leaf Jul 25, 2022, 2:01 AM

#

right

haughty mica Jul 25, 2022, 2:01 AM

#

You just can't write through deque while iter exists, because normally you could invalidate iter but since it is a function argument, you cannot

#

I think this might be fixed by making the iterator wrap a raw slice

tough leaf Jul 25, 2022, 2:02 AM

#

can you move the iteration logic all to a RawIter and then have the others just be thin wrappers over that

#

that turn a *mut T into whatever is valid for them

#

also iter::adapters::flatten::test_trusted_len_flatten still running

#

no sign of stopping

#

so i guess TODO: work out why that takes actually fucking hours to run under miri

#

let iter = [(), (), ()].iter().flat_map(|_| [(); 1000]);
assert_trusted_len(&iter);
assert_eq!(iter.size_hint(), (3000, Some(3000)));

#

ah

#

i guess that's probably it

haughty mica Jul 25, 2022, 2:11 AM

#

Test passes now

#

Yeah that's it

#

I commented with my explanation

#

Someone will have fun patching this

tough leaf Jul 25, 2022, 2:26 AM

#

Ohhh I think I get it. it's basically

pub fn foo(x: &u32, y: *mut u32) {
y.write(42);
}

where x and y alias?

#

and that's not okay because if you pass something as an argument it must hold until the return of the function?

haughty mica Jul 25, 2022, 2:38 AM

#

Yes

tender nimbus Jul 25, 2022, 7:21 AM

#

because we want funny llvm attributes ferrisWhen

neon tiger Jul 25, 2022, 9:07 AM

#

we do a little optimization

tender nimbus Jul 25, 2022, 9:12 AM

#

I was looking a bit at sled for entertainment because I've heard it's very cursed and found this gem of an issue https://github.com/spacejam/sled/issues/1407

GitHub

SIGSEGV on db.insert() · Issue #1407 · spacejam/sled

I get an (signal: 11, SIGSEGV: invalid memory reference) error when trying to insert into sled. I am using serde/bincode to get a Vec<u8> of my struct. Relevant code is: pub fn cache_...

neon tiger Jul 25, 2022, 9:13 AM

#

that is pretty funny ngl

#

the mem::transmute paw curls a finger as it passes the size check

tender nimbus Jul 25, 2022, 9:19 AM

#

hey fewwis can you eval ```rust
unsafe {
let mut vec = Vec::from_raw_parts(1 as *mut u8, 0, 10);
vec.push(1);
}

small apexBOT Jul 25, 2022, 9:19 AM

#

timeout: the monitored command dumped core
/playground/tools/entrypoint.sh: line 11:     7 Segmentation fault      timeout --signal=KILL ${timeout} "$@"

tender nimbus Jul 25, 2022, 9:19 AM

#

holy shit that's a segfault in Vec::push, better open an issue

neon tiger Jul 25, 2022, 9:20 AM

#

rustsec advisory incoming

#

all hands on deck, it's a 10

tender nimbus Jul 25, 2022, 9:22 AM

#

neon tiger the `mem::transmute` paw curls a finger as it passes the size check

a size check on transmute simply isn't enough
we also need a vibe check

neon tiger Jul 25, 2022, 9:26 AM

#

// in std::mem
enum Vibe {
  Inty,
  Pointery,
  Uninity
}

tender nimbus Jul 25, 2022, 9:26 AM

#

error[E0512]: cannot transmute between types of different vibes
 --> src/main.rs:3:5
  |
3 |     std::mem::transmute::<u64, &[u8; 8]>(0)
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
  |
  = note: source type: `u64` (inty vibes)
  = note: target type: `&[u8; 8]` (pointery vibes)

For more information about this error, try `rustc --explain E0512`.

#

this would be a great addition to our memory model, since it's very vibey already

tough leaf Jul 25, 2022, 10:58 AM

#

wait could we panic on any transmute from an integer type to a non-ZST reference

#

transmutes don't bless the reference with provenance

#

(it's fine for a ZST tho)

#

and if we're saying it's dereferencable....

tender nimbus Jul 25, 2022, 11:01 AM

#

tough leaf transmutes don't bless the reference with provenance

they should not but has this been decided?

tough leaf Jul 25, 2022, 11:06 AM

#

https://github.com/rust-lang/unsafe-code-guidelines/issues/286

GitHub

What about: Pointer-to-integer transmutes? · Issue #286 · rust-lang...

Transmuting pointers to integers (i.e., not going through the regular cast) is a problem. This is demonstrated by the following silly example: fn example(ptr: *const i32, cmp: usize) -> usiz...

#

this is about ptrtoint but it does say provenance is removed

tough leaf Jul 25, 2022, 11:07 AM

#

tender nimbus ```rust error[E0512]: cannot transmute between types of different vibes --> src...

and i'm fairly sure if we don't emit a inttoptr on a transmute, this is currently LLVM UB

neon tiger Jul 25, 2022, 11:08 AM

#

how do we compile from_exposed_addr?

#

given recent events, the answer is probably "wrongly" ferrisWhen but how should we compile it?

tough leaf Jul 25, 2022, 11:09 AM

#

llvm has inttoptr and ptrtoint instructions, doesn't it?

#

?llvmir

#![feature(strict_provenance)]

pub fn owo(x: usize) -> *const u8 {
  std::ptr::from_exposed_addr(x)
}

pub fn uwu(x: usize) -> *const u8 {
  unsafe { std::mem::transmute(x) }
}

small apexBOT Jul 25, 2022, 11:10 AM

#

target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

define i8* @_ZN7example3owo17h6e6f82da7fd68509E(i64 %x) unnamed_addr #0 {
  %0 = inttoptr i64 %x to i8*
  ret i8* %0
}

attributes #0 = { mustprogress nofree norecurse nosync nounwind nonlazybind readnone uwtable willreturn "probe-stack"="__rust_probestack" "target-cpu"="x86-64" }

!llvm.module.flags = !{!0, !1}

!0 = !{i32 7, !"PIC Level", i32 2}
!1 = !{i32 2, !"RtLibUseGOT", i32 1}

tough leaf Jul 25, 2022, 11:11 AM

#

hmmmm

neon tiger Jul 25, 2022, 11:11 AM

#

tough leaf llvm has inttoptr and ptrtoint instructions, doesn't it?

yes

tough leaf Jul 25, 2022, 11:11 AM

#

does a transmute do an inttoptr too?

tender nimbus Jul 25, 2022, 11:11 AM

#

hey fewwis can you llvmir ```rust
pub fn uwu(x: usize) -> *const u8 {
unsafe { std::mem::transmute(x) }
}

small apexBOT Jul 25, 2022, 11:11 AM

#

target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

define i8* @_ZN7example3uwu17h0d2aef5dcb2270caE(i64 %x) unnamed_addr #0 {
  %0 = inttoptr i64 %x to i8*
  ret i8* %0
}

attributes #0 = { mustprogress nofree norecurse nosync nounwind nonlazybind readnone uwtable willreturn "probe-stack"="__rust_probestack" "target-cpu"="x86-64" }

!llvm.module.flags = !{!0, !1}

!0 = !{i32 7, !"PIC Level", i32 2}
!1 = !{i32 2, !"RtLibUseGOT", i32 1}

tender nimbus Jul 25, 2022, 11:11 AM

#

yeah

tough leaf Jul 25, 2022, 11:11 AM

#

it DOES

#

sneaky fuck

neon tiger Jul 25, 2022, 11:11 AM

#

small apex ```llvm target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:1...

wait where's the uwu

tough leaf Jul 25, 2022, 11:11 AM

#

?llvmir

pub fn uwu(x: [usize;2]) -> [*const u8; 2] {
  unsafe { std::mem::transmute(x) }
}

pub fn owo(x: [usize;2]) -> [*const u8; 2] {
  x.map(|x| x as *const u8)
}

small apexBOT Jul 25, 2022, 11:11 AM

#

target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

%"unwind::libunwind::_Unwind_Exception" = type { i64, void (i32, %"unwind::libunwind::_Unwind_Exception"*)*, [6 x i64] }
%"unwind::libunwind::_Unwind_Context" = type { [0 x i8] }

define void @_ZN7example3uwu17h1bbbc5917c46399fE([2 x i8*]* noalias nocapture noundef writeonly sret([2 x i8*]) dereferenceable(16) %0, [2 x i64]* noalias nocapture noundef readonly dereferenceable(16) %x) unnamed_addr #0 {
  %1 = bitcast [2 x i64]* %x to i8*
  %2 = bitcast [2 x i8*]* %0 to i8*
  call void @llvm.memcpy.p0i8.p0i8.i64(i8* noundef nonnull align 8 dereferenceable(16) %2, i8* noundef nonnull align 8 dereferenceable(16) %1, i64 16, i1 false)
  ret void
}

define void @_ZN7example3owo17h3a66012a35f73094E([2 x i8*]* noalias nocapture noundef writeonly sret([2 x i8*]) dereferenceable(16) %0, [2 x i64]* noalias nocapture noundef readonly dereferenceable(16) %x) unnamed_addr #0 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* @rust_eh_personality {
  %1 = bitcast [2 x i64]* %x to <2 x i64>*
  %2 = load <2 x i64>, <2 x i64>* %1, align 8
  %3 = inttoptr <2 x i64> %2 to <2 x i8*>
  %4 = bitcast [2 x i8*]* %0 to <2 x i8*>*
  store <2 x i8*> %3, <2 x i8*>* %4, align 8, !alias.scope !2, !noalias !7
  ret void
}

declare void @llvm.memcpy.p0i8.p0i8.i64(i8* noalias nocapture writeonly, i8* noalias nocapture readonly, i64, i1 immarg) #1

declare noundef i32 @rust_eh_personality(i32, i32 noundef, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*) unnamed_addr #2

attributes #0 = { mustprogress nofree nosync nounwind nonlazybind uwtable willreturn "probe-stack"="__rust_probestack" "target-cpu"="x86-64" }
attributes #1 = { argmemonly mustprogress nofree nounwind willreturn }
attributes
```Output too large. Godbolt link: <https://godbolt.org/z/1sPb53fWM>

tough leaf Jul 25, 2022, 11:12 AM

#

neon tiger wait where's the uwu

removed probably

#

for being the exact same function as owo

neon tiger Jul 25, 2022, 11:12 AM

#

small apex ```llvm target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:1...

nice

tough leaf Jul 25, 2022, 11:12 AM

#

they'll point to the same function

tender nimbus Jul 25, 2022, 11:12 AM

#

deduplicated, yeah

tough leaf Jul 25, 2022, 11:12 AM

#

now

neon tiger Jul 25, 2022, 11:12 AM

#

so transmute already knows the types' vibes internally ferrisOwO

tough leaf Jul 25, 2022, 11:12 AM

#

is this okay

#

does this inttoptr

#

or is it literally

#

if you are a single pointer

#

then we'll inttoptr you

#

otherwise

#

you're on your own

#

use as

neon tiger Jul 25, 2022, 11:13 AM

#

what does bitcast do ferrisThonk

tender nimbus Jul 25, 2022, 11:14 AM

#

https://github.com/rust-lang/rust/blob/2f320a224e827b400be25966755a621779f797cc/compiler/rustc_codegen_ssa/src/mir/block.rs#L1573

tough leaf Jul 25, 2022, 11:14 AM

#

neon tiger what does `bitcast` do <:ferrisThonk:379064024865701898>

presumably a pointer cast

tender nimbus Jul 25, 2022, 11:14 AM

#

HACK(eddyb) LLVM doesn't like bitcasts between pointers and non-pointers.
lol

neon tiger Jul 25, 2022, 11:15 AM

#

tough leaf presumably a pointer cast

https://llvm.org/docs/LangRef.html#i-bitcast

#

yeah it looks like UB

#

to bitcast between ints and pointers

#

very cool hack thank you eddyb

tough leaf Jul 25, 2022, 11:19 AM

#

so basically, uwu takes two arguments, both pointers
a pointer to the start array as a [2 x i64]*, and a pointer to the return place as a [2 x i8*]*
we cast both of those pointers to i8* so we can pass them to memcpy, then we do the memcpy
then we return

#

so yeah

#

provenance is not kept

neon tiger Jul 25, 2022, 11:27 AM

#

I think this is related to the "LLVM IR can't implement memcpy" issue

#

to which the proposed solution are byte types

#

you could then emit bytecast for all transmutes and LLVM would do the right thing

tough leaf Jul 25, 2022, 11:31 AM

#

well, not quite

#

if you want to turn a usize into a *const u8 in a way that recovers exposed provenance, you need to actively do an operation on that usize

#

you can't just cast it

#

ptr <-> int transmutes aren't UB

but integers don't hold provenance

neon tiger Jul 25, 2022, 11:34 AM

#

tough leaf you can't just cast it

you could bytecast it

tough leaf Jul 25, 2022, 11:34 AM

#

does a bytecast do an implicit ptrtoint

neon tiger Jul 25, 2022, 11:34 AM

#

the proposed semantics are that bytecast from int to ptr does ptrtoint

tough leaf Jul 25, 2022, 11:34 AM

#

even for arrays

#

ah

#

isn't that kind of a footgun

neon tiger Jul 25, 2022, 11:35 AM

#

it's basically a device to select the right cast for you

tough leaf Jul 25, 2022, 11:36 AM

#

safety: U is not larger than T
unsafe fn transmute<T>(item: T) -> U {
  let mu = ManuallyDrop::new(item);
  let p = &mu as *const ManuallyDrop<T> as *const T as *const U;
  p.read_unaligned()
}

#

does this do a ptrtoint?

neon tiger Jul 25, 2022, 11:37 AM

#

depends on how you lower it

tough leaf Jul 25, 2022, 11:37 AM

#

where is the ptrtoint done though?

#

at the read_unaligned?

neon tiger Jul 25, 2022, 11:37 AM

#

probably yeah

tender nimbus Jul 25, 2022, 11:37 AM

#

i mean

tough leaf Jul 25, 2022, 11:38 AM

#

but the read_unaligned read a pointer

#

not an int

#

it doesn't have the int

neon tiger Jul 25, 2022, 11:38 AM

#

hmm?

#

oh, I see what you mean

tough leaf Jul 25, 2022, 11:38 AM

#

does every read_unaligned have to magically

tender nimbus Jul 25, 2022, 11:38 AM

#

you can transform this "addrof ptrcast read" into "a_real_transmute"
at which point there is only one place for it to happen

tough leaf Jul 25, 2022, 11:38 AM

#

do a ptrtoint

#

just in case

neon tiger Jul 25, 2022, 11:38 AM

#

you would not read int nor pointer

#

you'd read a b64

#

and bytecast it

tender nimbus Jul 25, 2022, 11:39 AM

#

but yeah

tough leaf Jul 25, 2022, 11:39 AM

#

so literally all memory access goes through bytes

neon tiger Jul 25, 2022, 11:39 AM

#

yes

#

unlike C, where only memory accesses at character type go through bytes ferrisWhen

#

Rust has no strict aliasing after all

tough leaf Jul 25, 2022, 11:40 AM

#

this doesn't seem sound in the general case
but i can't think of a counterexample

#

where it's literally impossible to confuse llvm into not inserting an inttoptr

#

even through, say, union reads

tough leaf Jul 25, 2022, 11:41 AM

#

neon tiger Rust has no strict aliasing after all

sure, but bytes have runtime types

#

not in the "typed memory" sense

#

but a uninit byte is different from a raw byte is different from a pointer byte

tender nimbus Jul 25, 2022, 11:41 AM

#

what are you talking about exacty 522?
AM semantics?
LLVM semantics?
rust to llvmir lowering?

tough leaf Jul 25, 2022, 11:42 AM

#

the proposal is to make LLVM always detect any form of ptr to int transmutes

#

as i understand it

neon tiger Jul 25, 2022, 11:42 AM

#

tough leaf but a uninit byte is different from a raw byte is different from a pointer byte

yes

tender nimbus Jul 25, 2022, 11:42 AM

#

at the AM level we have

enum Byte { Int(u8), Ptr(u8, Prov), Uninit }

the memory read_unaligned is reading from is of type Int
but we want a pointer, so we need to convert it into Ptr
converting - inttoptr

neon tiger Jul 25, 2022, 11:42 AM

#

that's exactly what the byte types are for

tough leaf Jul 25, 2022, 11:42 AM

#

what if i read 1 byte at a time

#

into some place

neon tiger Jul 25, 2022, 11:43 AM

#

to reflect the possibility of having integer, pointer and/or padding bytes in the same type

#

or poison etc.

#

the upshoot is that pointer types need not be able to carry integers and vice-versa

tough leaf Jul 25, 2022, 11:44 AM

#

so we're merging int and ptr bytes

#

ints can have provenance

neon tiger Jul 25, 2022, 11:44 AM

#

no, we're introducing a third type in the middle

tough leaf Jul 25, 2022, 11:44 AM

#

no, in the abstract machine

neon tiger Jul 25, 2022, 11:44 AM

#

exactly so integers don't need to have provenance

tender nimbus Jul 25, 2022, 11:44 AM

#

tough leaf ints can have provenance

no

#

this llvm byte is not an integer

tough leaf Jul 25, 2022, 11:44 AM

#

if i do a ptr to int transmute, honestly, i'd expect that to be a no-op

neon tiger Jul 25, 2022, 11:45 AM

#

what is a "no-op"?

tough leaf Jul 25, 2022, 11:45 AM

#

well

#

hm

#

i guess padding exists

tender nimbus Jul 25, 2022, 11:45 AM

#

ptr2int transmute is ```rust
match byte {
Ptr(addr, _prov) => Int(addr),
_ => unreachable!(),
}

neon tiger Jul 25, 2022, 11:45 AM

#

no actual physical bytes are being changed

#

so in that sense it is a no-op

#

a bunch of magic voodoo metastate is being changed

#

which we have decided we're fine with doing all over the place

#

I view the byte types as basically

#[non_exhaustive]
union Byte {
  Int,
  Pointer,
  Uninit,
  Poison,
  Undef,
  Padding,
  Null,
  Nil,
  NullButFromAnotherAddressSpace,
  MoreLLVMWaysToSpellUninit,
}

#

it's type erasure for typed memory

tender nimbus Jul 25, 2022, 12:33 PM

#

https://github.com/rust-lang/rust/issues/93881#issuecomment-1193987540 wtf is going on here

#

😵‍💫

#

the playground doesn't display any message

#

it has the exact same rustc version than i have (rustc 1.62.1 (e092d0b6b 2022-07-16))

#

(it doesn't display any message on nightly either)

haughty mica Jul 25, 2022, 1:59 PM

#

neon tiger given recent events, the answer is probably "wrongly" <:ferrisWhen:6964935883889...

As an intrinsic ferrisWhen

haughty mica Jul 25, 2022, 1:59 PM

#

tender nimbus <https://github.com/rust-lang/rust/issues/93881#issuecomment-1193987540> wtf is ...

That's still the bad error, no?

tender nimbus Jul 25, 2022, 2:00 PM

#

oh, the offsets

#

yeah

#

but what is going on on the playground

haughty mica Jul 25, 2022, 3:23 PM

#

Not sure, but there was recently a PR merged to add track_caller for Miri

#

Might be relevant?

tender nimbus Jul 25, 2022, 3:24 PM

#

maybe, no idea

#

well

#

if it's stable "recently" then maybe

haughty mica Jul 25, 2022, 4:13 PM

#

Not stable

#

Nightly, yesterday or the day before

tender nimbus Jul 25, 2022, 4:14 PM

#

hmm, that's weird, because the playground is broken on stable as well

haughty mica Jul 26, 2022, 11:37 PM

#

The good news is that nextest seems to be working
The bad news is that everything takes forever now

#

Some crates now take more than an hour

#

Maybe I should cap runtime per-crate as well as per-test

tender nimbus Jul 27, 2022, 7:56 AM

#

ferrisballSweat

west violet Jul 27, 2022, 2:57 PM

#

haughty mica The good news is that nextest seems to be working The bad news is that everythin...

Can you integrate JUnit output and github actions so that test failures look nice?

haughty mica Jul 27, 2022, 3:13 PM

#

Um what

haughty mica Jul 27, 2022, 3:14 PM

#

west violet Can you integrate JUnit output and github actions so that test failures look nic...

Why are you asking about JUnit

west violet Jul 27, 2022, 3:15 PM

#

Nextest has stuff for it

#

I assumed that's why you were using it for your miri thingy (does it have a better name?) so that you'd have machine-readable output

haughty mica Jul 27, 2022, 3:17 PM

#

Lol no

#

I'm using nextest because it has per-test timeouts and process isolation for tests

west violet Jul 27, 2022, 3:18 PM

#

I do realize that I didn't give any context for that lmao

#

I had an entire convo in my head and you got exactly none of it

haughty mica Jul 27, 2022, 3:18 PM

#

All I care about is the exit code and parsing Miri diagnostics

#

Also I want the website to look just like your terminal would if you ran the tests yourself

tender nimbus Jul 27, 2022, 3:20 PM

#

west violet I assumed that's why you were using it for your miri thingy (does it have a bett...

you wanted a good name for it saethlin, here's one ferrisOwO

west violet Jul 27, 2022, 3:20 PM

#

Everything was fine until the Miri nation attacked

haughty mica Jul 27, 2022, 3:25 PM

#

ablobthinking

#

Mirithon?

#

Because it runs for a long time

tender nimbus Jul 27, 2022, 3:28 PM

#

that sounds fun

west violet Jul 27, 2022, 4:02 PM

#

Milar (Mylar) because it:

Strengthens the ecosystem
Stretches programs/the machines its running on
Makes crates more reliable

haughty mica Jul 27, 2022, 4:05 PM

#

Hmmmmmmmmmmmmmmmmm

neon tiger Jul 27, 2022, 8:35 PM

#

UB-quitty

#

sees everything
helps you quit your UB habit

haughty mica Jul 27, 2022, 8:39 PM

#

Oooooooooooo

wraith wind Jul 28, 2022, 2:22 PM

#

Is https://github.com/petgraph/petgraph/pull/505 a good fix for https://miri.saethlin.dev/ub?crate=petgraph&version=0.6.2? I tried to make it verify the safety more statically. I'm not sure if I could safely make the slice splits _unchecked, although running the benchmarks seems to have a negligible speed difference between the original code and mine

tough leaf Jul 28, 2022, 2:26 PM

#

wraith wind Is <https://github.com/petgraph/petgraph/pull/505> a good fix for <https://miri....

the swap could probably be https://doc.rust-lang.org/std/primitive.slice.html#method.swap_with_slice ?

slice - Rust

A dynamically-sized view into a contiguous sequence, [T]. Contiguous here means that elements are laid out so that every element is the same distance from its neighbors.

#

see if that optimises to be good enough

wraith wind Jul 28, 2022, 2:27 PM

#

swap_with_slice is just an assert + swap_nonoverlapping, and the slice splitting already guarentees they don't overlap and are of equal length

tender nimbus Jul 28, 2022, 2:27 PM

#

a smaller fix would be to just do some pointer arithmetic on the first pointer to get the second pointer, but this works as well if it works

tough leaf Jul 28, 2022, 2:27 PM

#

so yeah, swap_with_slice should be fine

#

and completely remove the unsafe

wraith wind Jul 28, 2022, 2:27 PM

#

I did try pointer.add() but Miri still gave me the same error

tender nimbus Jul 28, 2022, 2:27 PM

#

#[cfg_attr(miri, ignore = "Can't open files with isolation")]

MIRIFLAGS="-Zmiri-disable-isolation"

tough leaf Jul 28, 2022, 2:27 PM

#

at least in the bit of the code anyways

tough leaf Jul 28, 2022, 2:27 PM

#

tender nimbus `#[cfg_attr(miri, ignore = "Can't open files with isolation")]` `MIRIFLAGS="-Zm...

or include_bytes!()

#

depends how big the file is

wraith wind Jul 28, 2022, 2:29 PM

#

tough leaf so yeah, swap_with_slice should be fine

I guess it just seems a bit redundant, although iirc there's no performance impact to never-taken branches 🤔

tender nimbus Jul 28, 2022, 2:29 PM

#

it's not quite "none"

#

but it shouldn't really matter here

wraith wind Jul 28, 2022, 2:30 PM

#

tough leaf depends how big the file is

> wc -c tests/res/*
1999999 tests/res/graph_1000n_1000e.txt
1999999 tests/res/graph_1000n_1000e_iso.txt
  19999 tests/res/graph_100n_100e.txt
  19999 tests/res/graph_100n_100e_iso.txt
4039996 total

wraith wind Jul 28, 2022, 2:30 PM

#

tender nimbus it's not quite "none"

I think I got that from https://blog.cloudflare.com/branch-predictor/, might have misinterpreted what they meant

tender nimbus Jul 28, 2022, 2:31 PM

#

it depends a lot, but often it is "none"

#

Takes too long to run in Miri
could you maybe make the graph a little smaller in miri? would that be fast enough?

tender nimbus Jul 28, 2022, 2:32 PM

#

tender nimbus > Takes too long to run in Miri could you maybe make the graph a little smaller ...

(if cfg!(miri) { small } else { big })

#

or using #[cfg(miri)]

wraith wind Jul 28, 2022, 2:33 PM

#

I wouldn't really know enough about the crate to change the test so that it still passes :/

#

tender nimbus Jul 28, 2022, 2:33 PM

#

ferrisBut i am clueless about gaphs as well

wraith wind Jul 28, 2022, 2:34 PM

#

I disabled that one because it was 16gb of RAM and 10 minutes in on my PC which I thought seemed a bit much

tender nimbus Jul 28, 2022, 2:34 PM

#

that is a lot indeed

#

but yeah try to do it safely as 522 mentioned

#

btw

#

secret rust trick

#

- node_adjacencies[pos..pos + old_node_capacity]
+ node_adjacencies[pos..][..old_node_capacity]

wraith wind Jul 28, 2022, 2:36 PM

#

Ooh

wraith wind Jul 28, 2022, 2:45 PM

#

wraith wind ``` > wc -c tests/res/* 1999999 tests/res/graph_1000n_1000e.txt 1999999 tests/re...

I looked away for a minute and miri ate 30gb of ram ferrisBut

tender nimbus Jul 28, 2022, 2:47 PM

#

ferrisForgor

pastel lily Jul 28, 2022, 2:54 PM

#

yea sounds right

wraith wind Jul 28, 2022, 2:58 PM

#

tender nimbus it depends a lot, but often it is "none"

Building in release it seems like the assertion gets eliminated completely any way ferristhumbsup

tender nimbus Jul 28, 2022, 2:58 PM

#

yeah, that's also often the case

ebon orbit Jul 29, 2022, 6:31 AM

#

tender nimbus ```diff - node_adjacencies[pos..pos + old_node_capacity] + node_adjacencies[pos....

I feel like there should be a clippy lint for this?

tender nimbus Jul 29, 2022, 6:33 AM

#

maybe

ebon orbit Jul 29, 2022, 2:09 PM

#

?clippy

fn main() {
    let list = [1, 2, 3, 4];
    dbg!(&list[1..1 + 2]);
    dbg!(&list[1..][..2]);
}

small apexBOT Jul 29, 2022, 2:09 PM

#

tough leaf Jul 29, 2022, 2:17 PM

#

it would be a subjective lint

#

i don't think it should be on by default

#

since i think it is somewhat more confusing

tender nimbus Jul 29, 2022, 2:19 PM

#

it's a little more confusing if you learn about it the first time

#

but it makes code a lot nicer once you know it

pastel lily Jul 29, 2022, 2:26 PM

#

does it behave worse for bounds checks

#

well

#

bounds checks it's still just a min and a max

#

but does LLVM get the same range info

#

does it understand that that's start..len

tender nimbus Jul 29, 2022, 2:29 PM

#

hey fewwis can you godbolt ```rust
pub fn uwu(a: &[u8], x: usize, y: usize) -> &[u8] {
&a[x..x + y]
}

pub fn owo(a: &[u8], x: usize, y: usize) -> &[u8] {
&a[x..][..y]
}

small apexBOT Jul 29, 2022, 2:29 PM

#

example::uwu:
        push    rax
        mov     rax, rcx
        add     rax, rdx
        jb      .LBB0_3
        cmp     rax, rsi
        ja      .LBB0_4
        add     rdi, rdx
        mov     rax, rdi
        mov     rdx, rcx
        pop     rcx
        ret
.LBB0_3:
        lea     rcx, [rip + .L__unnamed_1]
        mov     rdi, rdx
        mov     rsi, rax
        mov     rdx, rcx
        call    qword ptr [rip + core::slice::index::slice_index_order_fail@GOTPCREL]
        ud2
.LBB0_4:
        lea     rdx, [rip + .L__unnamed_1]
        mov     rdi, rax
        call    qword ptr [rip + core::slice::index::slice_end_index_len_fail@GOTPCREL]
        ud2

example::owo:
        push    rax
        mov     rax, rsi
        sub     rsi, rdx
        jb      .LBB1_3
        cmp     rsi, rcx
        jb      .LBB1_4
        add     rdi, rdx
        mov     rax, rdi
        mov     rdx, rcx
        pop     rcx
        ret
.LBB1_3:
        lea     rcx, [rip + .L__unnamed_2]
        mov     rdi, rdx
        mov     rsi, rax
        mov     rdx, rcx
        call    qword ptr [rip + core::slice::index::slice_start_index_len_fail@GOTPCREL]
        ud2
```Output too large. Godbolt link: <https://godbolt.org/z/Ts8hf847c>

pastel lily Jul 29, 2022, 2:29 PM

#

smh we should be able to have two uwus

tender nimbus Jul 29, 2022, 2:30 PM

#

basically the same

tender nimbus Jul 29, 2022, 2:30 PM

#

pastel lily smh we should be able to have two `uwu`s

make an rfc

#

"RFC 1234 More Uwu"

pastel lily Jul 29, 2022, 2:31 PM

#

this does look interesting

#

uwu has a check for the range being reversed and that the end is not OOB

tough leaf Jul 29, 2022, 2:31 PM

#

x + y could overflow

#

so the range can indeed be reversed

pastel lily Jul 29, 2022, 2:32 PM

#

yeah

tough leaf Jul 29, 2022, 2:32 PM

#

ah, so this is actually faster

#

in the general case?

#

well

#

hm

pastel lily Jul 29, 2022, 2:33 PM

#

oh it only needs to check the end if overflow doesn't happen I guess?

#

hm

#

what is it doing

tender nimbus Jul 29, 2022, 2:33 PM

#

tough leaf ah, so this is actually faster

clippy/perf we got it

pastel lily Jul 29, 2022, 2:33 PM

#

yeah

tough leaf Jul 29, 2022, 2:33 PM

#

but like

pastel lily Jul 29, 2022, 2:33 PM

#

uwu only does a range order (overflow) check and a end check

tough leaf Jul 29, 2022, 2:34 PM

#

this seems actually non-optimal

pastel lily Jul 29, 2022, 2:34 PM

#

owo does a start check for the first slice and an end check for the second

#

these are semantically different

#

that's really interesting though that it removes the start check entirely, because the end check also fails iff the range is in order

haughty mica Aug 1, 2022, 9:55 PM

#

Sushi man needs a bonk

#

Did a UB

tough leaf Aug 1, 2022, 9:56 PM

#

haughty mica Did a UB

👀

#

in regex?

#

or elsewhere

haughty mica Aug 1, 2022, 9:56 PM

#

bstr

#

Pre release at least

#

I noticed it on the site

#

The one on the site is just UCG 133 but there's another use outside provenance

tender nimbus Aug 1, 2022, 10:16 PM

#

melobonk

tough leaf Aug 9, 2022, 4:34 PM

#

mem::uninit FCW time

#

time to look up how the deprecated lint actually works

haughty mica Aug 9, 2022, 4:36 PM

#

Yup!

tough leaf Aug 9, 2022, 4:38 PM

#

since we also have to lint for let _ = std::mem::uninitialized so you can't just look for calls :(

#

ah well

#

now we just need to get sign-off on "uninit u8 is UB" that would be good i think

tender nimbus Aug 9, 2022, 4:39 PM

#

ferrisUwU

tough leaf Aug 9, 2022, 4:41 PM

#

hmmmm
or i could make deprecated take a "future_warning" parameter

#

which makes it a FCW

#

that's weird

#

i'll just make a new lint

haughty mica Aug 9, 2022, 6:04 PM

#

tough leaf now we just need to get sign-off on "uninit u8 is UB" that would be good i think

No you don't, just add the attribute and it will be fine

#

Then justify the rule with the attribute

#

This is how we've done the rest of Rust semantics why stop now

tender nimbus Aug 9, 2022, 6:05 PM

#

eh, why even need attributes or anything

#

just do it

#

like box

#

"safe code can't exercise this behaviour therefore it can't happen"

pastel lily Aug 9, 2022, 7:47 PM

#

justify
bold assumption any Rust semantics are actually justified

delicate crown Aug 13, 2022, 7:58 AM

#

If anyone's looking for something to review, the not-so-vaporlang Roc opened its repo and it's got a decent amount of unsafe

https://github.com/roc-lang/roc/search?q=unsafe&type=

GitHub

Search · unsafe · roc-lang/roc

Roc is a language for making delightful software. Contribute to roc-lang/roc development by creating an account on GitHub.

tender nimbus Aug 13, 2022, 8:10 AM

#

nothing obviously bad from skimming through it a little

delicate crown Aug 13, 2022, 8:23 AM

#

Yeah, though they also use unsafe_op_in_unsafe. Sketchiest things to me are some weird wasm deref of int macros, but I don't know wasm well enough to say since it's a weird arch. Also this weird MU swap
https://github.com/roc-lang/roc/blob/641daab02c90b446d8dfad302a8c57c6dfab9f0d/crates/roc_std/src/lib.rs#L155
I recreated this in Miri and it didn't complain with the default settings. But also this function is sort of useless for non-safety reasons and I may make a PR removing it.

GitHub

roc/lib.rs at 641daab02c90b446d8dfad302a8c57c6dfab9f0d · roc-lang/roc

Roc is a language for making delightful software. Contribute to roc-lang/roc development by creating an account on GitHub.

tender nimbus Aug 13, 2022, 8:26 AM

#

couldn't they just make a ManuallyDrop self and then move out of the field ferrisballSweat

#

but also, mem::forget is an anti pattern

haughty mica Aug 13, 2022, 2:58 PM

#

I'm sure they don't know that

tender nimbus Aug 13, 2022, 2:59 PM

#

yeah

#

no one does :(

neon tiger Aug 13, 2022, 3:09 PM

#

most people think it's an anti-pattern because it causes leaks but the cases where you don't leak are often even worse

tender nimbus Aug 13, 2022, 3:12 PM

#

it's an anti-pattern because whenever you need to forget something it's usually because you don't want its drop code to run
and that's often because it has been put into some kind of invalid state, or it needs to not invalidate other things
if it's in an invalid state, moving it around is sketchy. and if it needs to now invalidate other things, moving it around is sketchy.

neon tiger Aug 13, 2022, 3:16 PM

#

stable vec_into_raw_parts when ferrisSob

#

this is such an obvious API and the available alternatives are all terrible footguns

tender nimbus Aug 13, 2022, 3:19 PM

#

neon tiger stable `vec_into_raw_parts` when <:ferrisSob:586979927430135818>

ferrisCowboyRTX bikeshedding over stupid details

tender nimbus Aug 22, 2022, 5:10 AM

#

ferrisOwO

tender nimbus Aug 30, 2022, 11:48 AM

#

don't die ferrisSob

tender nimbus Sep 6, 2022, 7:16 AM

#

@haughty mica have you looked into hashbrown being broken under field retagging

tender nimbus Sep 6, 2022, 7:34 AM

#

oh lol it's actually very trivial

#

mem::forget ferrisClueless

#

#[forbid(clippy::mem_forget)] when ferrisPlead

#

at least in unsafe code ferrisballSweat

tender nimbus Sep 6, 2022, 9:01 AM

#

and now ci fails because of a rust windows bug ferrisClueless

haughty mica Sep 6, 2022, 1:58 PM

#

Do you have a fix for it?

#

I think I looked at it but never attempted a fix

coral trout Sep 6, 2022, 2:13 PM

#

Fixing sb errors: 15 minutes of staring at the error to understand SB, 1 minute of actual changes.

tender nimbus Sep 6, 2022, 2:14 PM

#

haughty mica Do you have a fix for it?

yes, mem::forget->ManuallyDrop<T> ferrisButGrimacing

tender nimbus Sep 6, 2022, 2:15 PM

#

coral trout Fixing sb errors: 15 minutes of staring at the error to understand SB, 1 minute ...

thanks to the nice diagnostics i immediately knew the fix. there was "pointer was invalidated by a retag" pointing at mem::forget ferrisBut

#

i've opened a pr already but pc-windows-gnu is currently broken prevent CI from passing

#

(the toolchain itself)

#

(chrisd opened up a fix pr on rust-lang/rust)

coral trout Sep 6, 2022, 2:16 PM

#

I did spin::SpinMutex yesterday, one of the issues was that and was easy, the other was related to locks holding mut-refs instead of pointers

#

Which took me a bit to understand when exactly you got overlap, but once it clicked I now feel like I understand the SB model better than I did

tender nimbus Sep 6, 2022, 2:18 PM

#

ferrisOwO

haughty mica Sep 6, 2022, 2:38 PM

#

Ah very cool

coral trout Sep 6, 2022, 2:55 PM

#

(Aside: I should run field-retags on my crates, I'm sus about at least one of them)

haughty mica Sep 6, 2022, 2:57 PM

#

If they're published you can just look at them on the site

#

More reason to publish your stuff 😉

tender nimbus Sep 6, 2022, 2:58 PM

#

how many crates do you run currently?

haughty mica Sep 6, 2022, 2:58 PM

#

All of them

coral trout Sep 6, 2022, 2:58 PM

#

How would I look for a specific crate? ctrl-f doesn't seem to do it :P

tender nimbus Sep 6, 2022, 2:59 PM

#

haughty mica All of them

oh lmao
that explains why my browser OOMs sometimes on the front page ferrisForgor

#

neat

haughty mica Sep 6, 2022, 2:59 PM

#

You probably need to resort to the main listing on https://miri.saethlin.dev

#

It is large

tender nimbus Sep 6, 2022, 2:59 PM

#

you probably want to curl and grep it

#

the browser will be very sad

#

https://miri.saethlin.dev/?crate=stuff&version=0.2.0 ferrisSus

#

why do so many of those error ferrisSob

#

~~worse error rate than crater~~

grim copper Sep 6, 2022, 3:56 PM

#

Is there an AnyMap alternative which is more sound that we could recommend?

tender nimbus Sep 6, 2022, 3:59 PM

#

type map?

#

there's typemap_rev

thorny venture Sep 6, 2022, 7:09 PM

#

tender nimbus there's typemap_rev

I fixed the hasher impl on mine BTW ferrisBased

tender nimbus Sep 6, 2022, 7:10 PM

#

thorny venture I fixed the hasher impl on mine BTW <:ferrisBased:920768834930094111>

good

thorny venture Sep 6, 2022, 7:11 PM

#

I temporarily enabled my admin powers to deploy it without asking anyone

tender nimbus Sep 6, 2022, 7:12 PM

#

cowwo

thorny venture Sep 6, 2022, 7:13 PM

#

It's fine. People trust me almost entirely with rust related things

#

I'm our Rust BDFL

tender nimbus Sep 6, 2022, 7:15 PM

#

conrad learned a new word ferrisClueless

thorny venture Sep 6, 2022, 7:18 PM

#

ferrisUwU

wraith wind Sep 6, 2022, 11:43 PM

#

Having a look at https://miri.saethlin.dev/ub?crate=minimal-lexical&version=0.2.1 - a lot of the errors come from calling .as_(mut_)ptr() through Deref:

impl ops::Deref for StackVec {
    type Target = [bigint::Limb];
    #[inline]
    fn deref(&self) -> &[bigint::Limb] {
        // SAFETY: safe since `self.data[..self.len()]` must be initialized
        // and `self.len() <= self.capacity()`.
        unsafe {
            let ptr = self.data.as_ptr() as *const bigint::Limb;
            slice::from_raw_parts(ptr, self.len())
        }
    }
}

Almost all of the errors go away if I just add .as_ptr() and .as_mut_ptr() on the struct itself, but I'm not sure if that's a good fix, or if there's an issue with the deref implementation

#

self.data being a [MaybeUninit<u64>; 62]

haughty mica Sep 7, 2022, 12:53 AM

#

No that sounds like the right thing to do to me

#

This is slice deref right?

#

That retags the whole slice which people often don't want

wraith wind Sep 7, 2022, 1:10 AM

#

Yeah

tender nimbus Sep 7, 2022, 5:35 AM

#

tender nimbus <https://miri.saethlin.dev/?crate=stuff&version=0.2.0> <:ferrisSus:9856184584211...

@haughty mica why do so many normal crates error out like that?

haughty mica Sep 7, 2022, 2:36 PM

#

HUNH

#

Good question

tender nimbus Sep 7, 2022, 2:43 PM

#

ferrisThink

haughty mica Sep 7, 2022, 2:43 PM

#

Holy shit that's a lot of those errors

#

Aaaaaaa

#

45 thousand

#

fuuck

tender nimbus Sep 7, 2022, 2:57 PM

#

ferrisForgor

haughty mica Sep 7, 2022, 3:11 PM

#

I should add a kind of dashboard page so that it's obvious when I screw up like this

haughty mica Sep 7, 2022, 4:15 PM

#

Good news is the ETA is only 17 hours

#

Oh wait 19 hours

#

22 hours

#

Back to 15

#

I really played myself here

tender nimbus Sep 7, 2022, 4:22 PM

#

what was the problem?

haughty mica Sep 7, 2022, 4:24 PM

#

No idea.

#

Some mistake I made weeks ago

tender nimbus Sep 7, 2022, 4:26 PM

#

ferrisPensive

haughty mica Sep 7, 2022, 4:26 PM

#

At a guess, I was dorking with the commit that I build Miri from and got too creative

#

Btw if you are a sicko you can view the logs of a crate with a URL like this, because this is just cloudfront+s3:
https://miri.saethlin.dev/logs/stuff/0.2.0.html

#

No need to load the big list of all crates, but also no ability to navigate

tender nimbus Sep 7, 2022, 4:31 PM

#

ferrisOwO

haughty mica Sep 8, 2022, 5:09 PM

#

"How has this crate been building for hours?"

╰ ➤ du -h target/debug/build/arpabet_cmudict-b9b99059bb1900c4/out/codegen.rs 
38M    target/debug/build/arpabet_cmudict-b9b99059bb1900c4/out/codegen.rs

tender nimbus Sep 8, 2022, 5:24 PM

#

ferrisConcern

haughty mica Sep 9, 2022, 9:22 PM

#

Just got a bunch of ICEs because rustdoc can't create a temp dir 😩

haughty mica Sep 10, 2022, 5:04 PM

#

Okay I think the above situation is finally fixed

tender nimbus Sep 10, 2022, 5:15 PM

#

ferrisUwU

#

https://miri.saethlin.dev/?crate=stuff&version=0.2.0 ferrisUwU

#

thank you

grim copper Sep 10, 2022, 8:54 PM

#

@haughty mica how often are the tests on the ub miri website updated?

haughty mica Sep 10, 2022, 8:59 PM

#

When I damn well feel like it

#

If you have a request for something just let me know and I can probably do it quickly

tender nimbus Sep 10, 2022, 8:59 PM

#

or when the run feels really bad and errors everything forcing you to rerun ferrisballSweat

haughty mica Sep 10, 2022, 8:59 PM

#

Yeah that happens

#

Usually I don't let things get older than a few days, but that's in terms of published crates not Miri versions

#

Running newly published crates is a lot easier than re-running the universe

grim copper Sep 10, 2022, 10:05 PM

#

haughty mica If you have a request for something just let me know and I can probably do it qu...

arc-swap has an update which hopefully fixes the data-race

haughty mica Sep 10, 2022, 10:40 PM

#

grim copper arc-swap has an update which hopefully fixes the data-race

Cool, but the maintainer hasn't published it

grim copper Sep 10, 2022, 10:40 PM

#

haughty mica Cool, but the maintainer hasn't published it

looks like they have - v 1.51

haughty mica Sep 10, 2022, 10:42 PM

#

1.5.1 has been up on the site for a while

#

The site is rarely more than a few days behind

grim copper Sep 10, 2022, 10:43 PM

#

haughty mica Sep 10, 2022, 10:43 PM

#

This again

grim copper Sep 10, 2022, 10:43 PM

#

oh, https://miri.saethlin.dev/no-sb/ub is different

tender nimbus Sep 10, 2022, 10:43 PM

#

no hashbrown release yet ferrisPensive

haughty mica Sep 10, 2022, 10:43 PM

#

Oh yeah no-sb/ub is very different

tender nimbus Sep 10, 2022, 10:43 PM

#

grim copper oh, https://miri.saethlin.dev/no-sb/ub is different

that one is old

haughty mica Sep 10, 2022, 10:43 PM

#

the no-sb page is very out of date, I have like never updated it lol

#

I'll update it sooon

tender nimbus Sep 10, 2022, 10:44 PM

#

it would be cool if there was some kind of "state" at least for the top crates like "fixed on master", "pr open" or whatever
but that would be effort to maintain ferrisballSweat

haughty mica Sep 10, 2022, 10:45 PM

#

It would be a crazy amount of CPU work to do

#

I spend hardly any time maintaining the code for this thing, but like if you saw what this does to my power bill

grim copper Sep 10, 2022, 10:45 PM

#

could that be done client-side?

haughty mica Sep 10, 2022, 10:45 PM

#

Do what on client side?

grim copper Sep 10, 2022, 10:45 PM

#

I guess effort though

#

looking up the state of the repo / crate

haughty mica Sep 10, 2022, 10:46 PM

#

Seeing if it's fixed would involve running Miri

grim copper Sep 10, 2022, 10:46 PM

#

not seeing if it's fixed, more just if it's out of date (there's a new version released)

haughty mica Sep 10, 2022, 10:46 PM

#

Hm

#

That might be possible

tender nimbus Sep 10, 2022, 10:47 PM

#

for example, what's the status of the top ub crates

grim copper Sep 10, 2022, 10:48 PM

#

inb4 compile miri to wasm and run it client side

tender nimbus Sep 10, 2022, 10:48 PM

#

help: <5091> was created by a SharedReadOnly retag at offsets [0x18..0x30]
  --> src/lib.rs:156:42
   |
51 |                 ptr::copy_nonoverlapping(&hole_guard.v[index + 1], &mut hole_guard.v[index], 1);
   |                                          ^^^^^^^^^^^^^^^^^^^^^^^^
help: <5091> was later invalidated at offsets [0x0..0x60] by a Unique retag
  --> src/lib.rs:156:73
   |
51 |                 ptr::copy_nonoverlapping(&hole_guard.v[index + 1], &mut hole_guard.v[index], 1);

lmao classic (scopeguard)

haughty mica Sep 10, 2022, 10:49 PM

#

grim copper inb4 compile miri to wasm and run it client side

This is probably technically possible

tender nimbus Sep 10, 2022, 10:50 PM

#

it will only be a little bundle

#

just a few megabytes

#

ferrisClueless

haughty mica Sep 10, 2022, 10:51 PM

#

The biggest obstacle is that Miri dynamically links to the rustc libraries

wraith wind Sep 10, 2022, 10:51 PM

#

Exploit the users browser and upload + run miri natively

haughty mica Sep 10, 2022, 11:20 PM

#

@grim copper updated now

grim copper Sep 10, 2022, 11:20 PM

#

haughty mica <@119943947043864576> updated now

cheers

haughty mica Sep 10, 2022, 11:20 PM

#

Still working through the rest of the crates

#

eta 10 hours, but arc-swap is done

grim copper Sep 10, 2022, 11:20 PM

#

sweet, so csv looks like the next top contender

haughty mica Sep 10, 2022, 11:20 PM

#

That one is already spoken for, sadly

tender nimbus Sep 10, 2022, 11:21 PM

#

grim copper sweet, so csv looks like the next top contender

updated on master

grim copper Sep 10, 2022, 11:21 PM

#

ah

tender nimbus Sep 10, 2022, 11:21 PM

#

just like hashbrown ferrisWhen

haughty mica Sep 10, 2022, 11:22 PM

#

plotters though

#

That is big sus

tender nimbus Sep 10, 2022, 11:27 PM

#

I can't find it, can you send a link?

haughty mica Sep 10, 2022, 11:35 PM

#

https://miri.saethlin.dev/no-sb/ub?crate=plotters&version=0.3.4

tender nimbus Sep 10, 2022, 11:36 PM

#

ah, the no-sb

haughty mica Sep 10, 2022, 11:36 PM

#

Yeah we get timeout/OOM with SB on

#

I am working on the tag GC right now

tender nimbus Sep 10, 2022, 11:37 PM

#

plotters ferrisballSweat

#

what ferrisballSweat

#

why is their rgb pixel doing unaligned reads

haughty mica Sep 10, 2022, 11:39 PM

#

There's a

#[allow(clippy::many_single_char_names, clippy::cast_ptr_alignment)]

Above the function

tender nimbus Sep 10, 2022, 11:41 PM

#

oh they also do unsound tuple transmutes

#

of course

#

this crate is pretty sus

#

overall

#

ferrisballSweat

haughty mica Sep 10, 2022, 11:43 PM

#

Well, that's why the site exists 🙂

#

Now we know 🙂

#

And maybe the author can too 🙂

tender nimbus Sep 10, 2022, 11:43 PM

#

ferrisOwO

#

maybe someone should fix all their unsafe code ferrisClueless

grim copper Sep 10, 2022, 11:55 PM

#

I'm more worried about the alloc-[no-]stdlib crates

#

looks like they're used in dropbox and they're effectively just casting uninit memory

#

and brotli ferrisballSweat

#

at least plotting is not usually connected to the internet

haughty mica Sep 10, 2022, 11:57 PM

#

I think I fixed the alloc-no-stdlib crates? And they just didn't publish?

#

Or maybe not

grim copper Sep 10, 2022, 11:57 PM

#

https://github.com/dropbox/rust-alloc-no-stdlib/issues/12

GitHub

Tests contain UB: Use of uninitialized memory · Issue #12 · dropbox...

These lines both read the contents of a region allocated with malloc without initializing it. They can both be detected by running cargo miri test. rust-alloc-no-stdlib/tests/lib.rs Line 395 in 487...

#

opened jan 30

haughty mica Sep 10, 2022, 11:58 PM

#

What's this abut brotli?

#

Oh maybe I never fixed it

grim copper Sep 10, 2022, 11:58 PM

#

haughty mica What's this abut brotli?

looks like brotli is in its reverse dependencies

haughty mica Sep 10, 2022, 11:58 PM

#

Ah I see

#

Well from what I recall this is UB in the tests, not the actual code

#

Same as scopeguard

grim copper Sep 10, 2022, 11:59 PM

#

yeah, but the test looks like its just using one of the allocators provided by the crate

#

allocate array -> write to some of it -> read from it -> UB

haughty mica Sep 11, 2022, 12:00 AM

#

Yes?

#

So you're saying this has a soundness issue?

grim copper Sep 11, 2022, 12:01 AM

#

seems like it?

#

looks like it can leak data from the stack

#

allocate array -> don't write to it -> read from it -> data leak

#

no sure why the other tests didn't fail though, so maybe there's some condition to it

#

I haven't looked at the details

#

oh wait the failing case is on the heap not the stack

#

even worse

#

eg. if you can read uninit heap memory, you could eg. read private keys out of the process that have been deallocated

haughty mica Sep 11, 2022, 12:07 AM

#

If you feel strongly about this, talk to RustSec

#

The group is bigger and more active recently, I'm sure if you open an issue on rustsec/advisory-db at least pinkforest will engage with you

grim copper Sep 11, 2022, 12:08 AM

#

I'll have to investigate it more thorougly, which I may be able to do a bit tomorrow

#

I'd like to at least have a somewhat working exploit POC before I go to rustsec

haughty mica Sep 11, 2022, 12:09 AM

#

Well if it's an unsound library, exploits are not really a thing?

grim copper Sep 11, 2022, 12:09 AM

#

???

#

what u mean?

#

also dropbox has a bug bounty, so if I can make something juicy I may be able to payout

haughty mica Sep 11, 2022, 12:12 AM

#

A very unsound library like this may be used in a way which is completely correct

#

And in fact that tends to be the common case

grim copper Sep 11, 2022, 12:13 AM

#

right yeah. An exploit would just involve using the library in a way that resembles a real-world scenario and leaking some simulated info

haughty mica Sep 11, 2022, 12:14 AM

#

I don't think that would alter anyone's assessment of this issue

#

Everyone in rustsec will understand that this is use of uninitialized memory, which can be used to leak secrets

grim copper Sep 11, 2022, 12:15 AM

#

yeah I suppose

#

ok, I might write it up on rustsec tomorrow

haughty mica Sep 11, 2022, 12:15 AM

#

And it is unlikely that Dropbox will care, unless you can actually make one of their services leak a secret

tender nimbus Sep 11, 2022, 8:37 AM

#

File a rust sec and get 10.0 critical because "we don't know if it's misused like this and have to assume the worst"

#

ferrisClueless

haughty mica Sep 12, 2022, 12:12 AM

#

Okay no-sb/ub is up to date again with the top 10k crates

#

Wild I feel like there are at least 3 buffer overflows in here

tender nimbus Sep 12, 2022, 3:48 AM

#

simd again?

haughty mica Sep 12, 2022, 1:19 PM

#

No

#

At least one of them is a randomize-layout situation

tender nimbus Sep 12, 2022, 1:52 PM

#

ferrisSus

haughty mica Sep 12, 2022, 1:54 PM

#

People do UB for strange reasons

#

I always run these tests with randomize-layout

#

And I'm looking for invalid pointer offset errors

#

Because any buffer overflow will probably be one of those before you get to the OOB access

tender nimbus Sep 12, 2022, 2:10 PM

#

lol

west violet Sep 12, 2022, 4:23 PM

#

haughty mica At least one of them is a randomize-layout situation

Wait, I found something?

haughty mica Sep 12, 2022, 4:37 PM

#

Yeah randomize-layout turns up a fair bit of smelly code

west violet Sep 12, 2022, 4:46 PM

#

It was all worth it

tough leaf Sep 12, 2022, 4:50 PM

#

now make it randomize (T,)

#

(maybe don't)

haughty mica Sep 12, 2022, 4:51 PM

#

Did the padding thing ever land? I don't think so right

tough leaf Sep 12, 2022, 4:53 PM

#

https://github.com/rust-lang/rust/pull/97861 ?

GitHub

Added randomized field padding to `-Z randomize-layout` by Kixiron ...

Adds randomized padding before each struct field in valid structs when -Z randomize-layout is enabled, the maximum amount of padding is decided by the -Z max-layout-multiple=<MULTIPLE> flag.
...

tender nimbus Sep 12, 2022, 4:55 PM

#

now make it randomize fat pointers (DO NOT)

west violet Sep 12, 2022, 4:59 PM

#

It does, actually

#

I got stuck on some funky ci error at some point

#

I was actually just looking at it

tender nimbus Sep 12, 2022, 5:00 PM

#

wait what

#

doesn't that break std ptr::metadata

west violet Sep 12, 2022, 5:01 PM

#

No

tender nimbus Sep 12, 2022, 5:01 PM

#

why

west violet Sep 12, 2022, 5:02 PM

#

...because it doesn't?

tender nimbus Sep 12, 2022, 5:02 PM

#

but I thought std does union transmutes of fat pointers

tough leaf Sep 12, 2022, 5:11 PM

#

tender nimbus but I thought std does union transmutes of fat pointers

ig it's fine as long as you ensure that union has the same layout as fat pointers?

#

which uhhh

tender nimbus Sep 12, 2022, 5:43 PM

#

ferrisballSweat

#

also breaking fat pointers will break crates like dyn-clone

#

ferrisballSweat

tough leaf Sep 12, 2022, 5:44 PM

#

yeah i think fat pointer layout is something we just gotta specify

#

well

#

what is the current layout

tender nimbus Sep 12, 2022, 5:44 PM

#

yes

#

OTOH

#

It doesn't really matter

#

one ptr metadata is out

tough leaf Sep 12, 2022, 5:45 PM

#

?eval

let x: [usize; 2] = unsafe { std::mem::transmute(&[0] as &[i32]) };
x

small apexBOT Sep 12, 2022, 5:45 PM

#

     Running `target/debug/playground`

[93972223787100, 1]

tender nimbus Sep 12, 2022, 5:45 PM

#

I mean is de facto specified we won't change it and no one will

tough leaf Sep 12, 2022, 5:45 PM

#

grrrr

#

yay

#

okay

#

it's (ptr, len)

tender nimbus Sep 12, 2022, 5:46 PM

#

ferrisOwO

haughty mica Sep 12, 2022, 5:47 PM

#

It matters for all the old code that won't be upgraded

tender nimbus Sep 12, 2022, 5:48 PM

#

true

#

it will become less important but there's still no reason to change it

#

so Let's Specify It
(Someone should write an RFC but it will not be me because oh god it would be worse than offsetof)

haughty mica Sep 12, 2022, 5:56 PM

#

Lol

#

I'm not sure it would be

#

At the very least, if the same person would appear on my RFC I won't be patient at all with them

tender nimbus Sep 12, 2022, 6:05 PM

#

a lot of people have been told that unspecified layout of repr(Rust) is great (and it is great indeed!)
i would feel that many people would immediately be opposed to specify it - though if the RFC did a good job of explaining the reasons behind it it could work

west violet Sep 12, 2022, 6:12 PM

#

Specifying all rust layouts would be an exceedingly bad idea

#

Specifying slices/str, no brainer

#

Trait objects, ehhhhhh

#

We still haven't figured out trait objects fully

tender nimbus Sep 12, 2022, 6:17 PM

#

trait objects are the interesting thing that people actually rely on though

#

slice fat pointers don't really make sense, we have from_raw_parts/x as *const _, x.len()

#

i mean we could of course specify them to be a repr c tuple but i don't think that's worth actually spending effort on since i dont think people actually rely on that

tender nimbus Sep 12, 2022, 6:20 PM

#

west violet We still haven't figured out trait objects fully

yet we won't change the layout of dyn fat pointers because changing it would be a Bad Idea and Break A Lot Of Things

haughty mica Sep 12, 2022, 7:27 PM

#

Oh look it's the same time as with C++ ABI

neon tiger Sep 12, 2022, 8:58 PM

#

tough leaf what is the current layout

#[repr(C)]
pub struct TraitObject {
    pub data: *mut (),
    pub vtable: *mut (),
}

#

slice is same except vtable is len: usize

ruby jacinth Sep 12, 2022, 9:09 PM

#

more like rust struct Ptr<M: Metadata> { data: *mut (), meta: M, }

#

if its a slice, that meta is a usize, if its a trait object a ptr to vtable, if it is a normal pointer it's zero sized

west violet Sep 17, 2022, 8:13 PM

#

Oh lordy me

#

This is 100% ub https://github.com/GraphiteEditor/Graphite/blob/master/node-graph/borrow_stack/src/lib.rs

GitHub

Graphite/lib.rs at master · GraphiteEditor/Graphite

2D raster & vector editor that melds traditional layers & tools with a modern node-based procedural workflow. - Graphite/lib.rs at master · GraphiteEditor/Graphite

#

They alloc a pointer, throw it into a Box::from_raw() and then pop() drops a Box::from_raw() pointing into the original allocation

#

It also decrements the length after deallocating the element which seems uh… wrong

tender nimbus Sep 17, 2022, 8:18 PM

#

decrementing the length afterwards is ferrisballSweat

tender nimbus Sep 17, 2022, 8:18 PM

#

west violet They `alloc` a pointer, throw it into a `Box::from_raw()` and then `pop()` drops...

where's the problem here?

west violet Sep 17, 2022, 8:19 PM

#

There’s a Box<[T; N]> and it basically does drop(Box::from_raw(&boxed_array[n]))

tender nimbus Sep 17, 2022, 8:20 PM

#

so it loses write provenance?

west violet Sep 17, 2022, 8:20 PM

#

So you can’t deallocate a portion of an allocation

#

You also can’t write to said deallocated portion

tender nimbus Sep 17, 2022, 8:20 PM

#

ohhhh... it deallocates a suballocation????

#

wtf

west violet Sep 17, 2022, 8:21 PM

#

Ye

#

I have literally no idea how this worked well enough to pass

tender nimbus Sep 17, 2022, 8:22 PM

#

1.7k stars ferrisConcern

#

also ironic that the module is called borrow_stack

west violet Sep 17, 2022, 8:26 PM

#

It’s almost poetic

haughty mica Sep 17, 2022, 8:32 PM

#

Where are the tests

#

L M A O they have the default test still in the file

#

Empty safety comments too thonk

#

Wait a fucking second

#

What does this even do

#

I feel like that totally-not-a-dealloc is trying to call the drop impl

#

All that code that uses this is commented-out thonk

tender nimbus Sep 17, 2022, 8:39 PM

#

ferrisSus

west violet Sep 18, 2022, 12:09 AM

#

Yah, I’m not really even sure what this could even do

#

Popping doesn’t give you the value back

tough leaf Sep 19, 2022, 1:20 PM

#

west violet Popping doesn’t give you the value back

pop isn't even called anywhere

#

(having a look at this now)

#

nor is get...

#

is this all just dead code

#

yes

tender nimbus Sep 26, 2022, 2:26 PM

#

@haughty mica lmao i love the favicon

haughty mica Sep 26, 2022, 2:29 PM

#

Thanks

#

I need to change the title too

tender nimbus Sep 26, 2022, 2:29 PM

#

ferrisballSweat

#

corro

#

cowwo would also be great

haughty mica Sep 26, 2022, 2:30 PM

#

Doesn't read well

#

Already looks bad just in Discord

tender nimbus Sep 26, 2022, 2:42 PM

#

yeah it's a little small

#

ferrisball is great

haughty mica Sep 29, 2022, 3:40 AM

#

It is very funny to me how some C dependencies have built setups that do not handle cgroups correctly

#

Yes, I do in fact have 64 cores under there, but if you spawn 64 processes you are going to get them all scheduled onto one core and you're going to like it

tender nimbus Sep 29, 2022, 5:12 AM

#

their build setup may be older than cgroups and they've never bothered to fix it ferrisClueless

neon tiger Sep 29, 2022, 1:02 PM

#

blindly spawning nproc many threads is a time-honored tradition ferrisClueless

tender nimbus Sep 29, 2022, 1:04 PM

#

wait does nproc not respect cgroups

#

Print the number of processing units available to the current process, which may be less than the number of online processors

tough leaf Sep 29, 2022, 1:05 PM

#

tender nimbus > Print the number of processing units available to the current process, which m...

unless they're using --all

tender nimbus Sep 29, 2022, 1:06 PM

#

why the fuck would a build script do that

#

(I should not be asking this question)

tough leaf Sep 29, 2022, 1:09 PM

#

or they're not using nproc

#

and just fuckin guessing

#

based on... whatever

haughty mica Sep 29, 2022, 4:59 PM

#

I could be writing down the crates that do this but eh

neon tiger Sep 30, 2022, 7:50 AM

#

tough leaf based on... whatever

yeah

#

there's a million ways to do this wrong and ignore cgroups

#

parse /proc/cpuinfo or whatever

tender nimbus Sep 30, 2022, 8:08 AM

#

just parse some text files ferrisUwU

neon tiger Sep 30, 2022, 9:41 AM

#

linux moment

tender nimbus Sep 30, 2022, 10:14 AM

#

average unix philosopher

tender nimbus Oct 1, 2022, 5:18 PM

#

https://github.com/rust-lang/rust-clippy/pull/9572

GitHub

Add `as_ptr_cast_mut` lint by Nilstrieb · Pull Request #9572 · rust...

This lint detects calls to a &self-taking as_ptr method, where the result is then immediately cast to a *mut T. Code like this is probably invalid, as that pointer will not have write permissio...

#

ferrisThink

ruby jacinth Oct 1, 2022, 5:27 PM

#

great, yet another lint to ignore ferrisHotTake

tender nimbus Oct 1, 2022, 5:28 PM

#

ruby jacinth great, yet another lint to ignore <:ferrisHotTake:985618456584020048>

bonkmelo

ruby jacinth Oct 1, 2022, 5:34 PM

#

tender nimbus <:bonkmelo:827237768157331496>

if i have ```rust
struct Blah(*mut ());

impl Blah{
fn as_ptr(&self) -> *mut () {
self.0
}
}and dorust
blah.as_ptr() as *mut u8

tender nimbus Oct 1, 2022, 5:36 PM

#

no

#

it needs to return *const ()

haughty mica Oct 1, 2022, 5:52 PM

#

English is not your first language right

#

I can't explain why but your tense choices here are odd

tender nimbus Oct 1, 2022, 6:00 PM

#

haughty mica English is not your first language right

yes

tender nimbus Oct 1, 2022, 6:01 PM

#

haughty mica I can't explain why but your tense choices here are odd

maybe they are wrong

haughty mica Oct 1, 2022, 6:02 PM

#

Since as_ptr took a &self, the pointer won't have write permissions

In English it is common to say

Since as_ptr takes a &self, the pointer won't have write permissions

tender nimbus Oct 1, 2022, 6:02 PM

#

yeah that does sound weird

haughty mica Oct 1, 2022, 6:02 PM

#

I guess the idea is that you haven't made the call yet, so it's not right to use the past tense

#

If you were interrupting execution in the middle of the call with an error, your tenses would be correct

tender nimbus Oct 1, 2022, 6:03 PM

#

well, you have made the call when you're doing the cast
but I guess takes makes more sense

haughty mica Oct 1, 2022, 6:03 PM

#

You wrote the call, but you're not executing it

tender nimbus Oct 1, 2022, 6:03 PM

#

yeah

haughty mica Oct 1, 2022, 6:57 PM

#

Why do people have crates which hork in crater and dump gigabytes of logs

#

WHY

#

I'm uploading ASan logs and it's going very slow because


└── 100.00% [10.28 GB] ── logs
   ├── 51.55% [5.3 GB] ── clacks_mtproto
   ├── 11.69% [1.2 GB] ── pleingres
   ├── 4.68% [480.47 MB] ── ap_kcp
   ├── 0.56% [57.76 MB] ── goose
   ├── 0.35% [35.84 MB] ── braque
   ├── 0.32% [32.97 MB] ── cifar_10_loader
   ├── 0.29% [30.09 MB] ── noise_search_deps_librocksdb-sys
   ├── 0.21% [21.38 MB] ── asserhttp
   ├── 0.14% [14.27 MB] ── injrs
   ├── 0.13% [13.17 MB] ── mqf
   └── 0.10% [10.7 MB] ── fil_builtin_actors_bundle

tender nimbus Oct 1, 2022, 6:58 PM

#

ferrisConcern

haughty mica Oct 1, 2022, 6:59 PM

#

At least I only need to upload this once because the crate is dead

tender nimbus Oct 1, 2022, 7:01 PM

#

good.

haughty mica Oct 1, 2022, 7:36 PM

#

Ok here we go https://miri.saethlin.dev/asan/ub

#

It needs a LOT of TLC still

#

For example, filtering out all the false positives due to custom stack mapping

#

And the "requested allocation size" crashes

#

And I need backtraces and diagnosis for the SIGILL crashes

#

oops: https://miri.saethlin.dev/asan/ub?crate=indexed&version=0.2.0

#

Bad diagnosis

#

Anyway, someone should probably report all these security vulnerabilities ferrisClueless

tender nimbus Oct 1, 2022, 7:40 PM

#

ferrisClueless

haughty mica Oct 1, 2022, 7:40 PM

#

If someone scrapes this with a bot and reports every crate that uses loom OMEGALULiguess

tender nimbus Oct 1, 2022, 7:42 PM

#

time to open like 20 issues on loom ferrisClueless

#

https://miri.saethlin.dev/asan/ub?crate=integer-encoding&version=3.0.4 hmm, why is this failing??

haughty mica Oct 1, 2022, 7:44 PM

#

Miri knows: https://miri.saethlin.dev/ub?crate=integer-encoding&version=3.0.4

tender nimbus Oct 1, 2022, 7:44 PM

#

ah

#

i was testing it on master lol

haughty mica Oct 1, 2022, 7:44 PM

#

What do you mean by that?

tender nimbus Oct 1, 2022, 7:44 PM

#

i tested miri on their master branch

#

which passed

haughty mica Oct 1, 2022, 7:44 PM

#

Oh jorge fixed it already

#

... maybe

#

Anyway, the site is still valuable as a "do not use" flag

#

If they publish the fix, I'll update the site

tender nimbus Oct 1, 2022, 7:46 PM

#

ferrisOwO

#

https://github.com/max-ym/safe-uninit/blob/f6d14ca981e847f57cbdeaeba0cc2206e51b0bb2/src/foreign_core.rs#L90 lol, lmao

tough leaf Oct 1, 2022, 7:53 PM

#

wait why are floats commented out

#

lol

haughty mica Oct 1, 2022, 7:57 PM

#

lul

tender nimbus Oct 1, 2022, 7:57 PM

#

it's unmaintained has a rustsec and no one uses it so i guess its fine

haughty mica Oct 1, 2022, 7:57 PM

#

Oh I should add the init checks

#

Lot more crates about to apppear 💀

#

543 crates in the list now

#

Only 218 of those are uninit issues

#

Yeah I'm changing the URL

#

Hope nobody posted URLs anywhere yet ferrisClueless

tender nimbus Oct 1, 2022, 8:11 PM

#

which ones?

tough leaf Oct 1, 2022, 8:12 PM

#

also
maybe have the root page be a landing page

haughty mica Oct 1, 2022, 8:12 PM

#

The asan ones

tough leaf Oct 1, 2022, 8:12 PM

#

with some text about what the site is doing and what the links are

haughty mica Oct 1, 2022, 8:12 PM

#

Yes

#

Yes yes yes

#

This has been on my mind for a while and I would welcome a PR or a writeup by someone else

#

I just haven't gotten to it

tough leaf Oct 1, 2022, 8:12 PM

#

because / being "lol fuck your browser" is somewhat annoying

#

ye fair

haughty mica Oct 1, 2022, 8:13 PM

#

Is it still "lol fuck your browser" though

tough leaf Oct 1, 2022, 8:13 PM

#

haven't tried too recently

#

i'll try on mobile

#

surely it will perform well

haughty mica Oct 1, 2022, 8:14 PM

#

Oh dear

#

I just tried it... no sign of loading

#

😩

tender nimbus Oct 1, 2022, 8:14 PM

#

my browser (chrome) is feeling fucked

#

it's actually kinda usable though

#

just slow

haughty mica Oct 1, 2022, 8:14 PM

#

Yeah you can thank Zurr for that

tender nimbus Oct 1, 2022, 8:14 PM

#

after waiting a few seconds to load

haughty mica Oct 1, 2022, 8:40 PM

#

Furiosly refreshing my URL waiting for AWS infrastructure to warm up

thorny venture Oct 1, 2022, 8:41 PM

#

~~furiously refreshing my email waiting for AWS job offer to come through~~

haughty mica Oct 1, 2022, 8:47 PM

#

Ah it turns out I messed up in multiple ways

#

Between my 0% CPU usage, open window, and lack of pants, it's getting kinda cold in here now ferrisHmm

thorny venture Oct 1, 2022, 8:54 PM

#

Turn up that CPU. Also, "lack of pants" is a lot more concerning in British English

haughty mica Oct 1, 2022, 8:58 PM

#

I'm quite aware of the different meaning in British English ferrisWink

tender nimbus Oct 1, 2022, 8:59 PM

#

ferrisSus

haughty mica Oct 1, 2022, 9:02 PM

#

ok my feet are kinda yellow, definitely need to get dressed

tender nimbus Oct 1, 2022, 9:02 PM

#

ferrisConcern

haughty mica Oct 1, 2022, 9:03 PM

#

I'm remembering that my "simple static site" actually involves a buttload of different AWS stuff

tender nimbus Oct 1, 2022, 9:04 PM

#

it's web scale

haughty mica Oct 1, 2022, 9:26 PM

#

FINALLY

#

https://asan.saethlin.dev/ub

tender nimbus Oct 1, 2022, 9:27 PM

#

https://tysan.saethlin.dev/ub when ferrisClueless

haughty mica Oct 2, 2022, 11:47 PM

#

I just stuck a few panics into parsing paths where some terminal escape programs are like "fuck it don't handle this" and holy shit the strange stuff that's in these logs I have

#

For example, why does this one build script print environment variables,
Except when there are spaces,
It replaces them with 0x1F, the Unit Separator

#

I'm learning about most of these by reading the alacritty source code, because it helpfully documents what these all are then fully ignores most of them

#

Swap foreground and background colors; inconsistent emulation
Sounds like I can ignore it then!

tender nimbus Oct 3, 2022, 3:52 AM

#

ferrisballSweat

thorny venture Oct 3, 2022, 10:00 AM

#

oh fuck oh fuck oh fuck

#

I decided to run my tests through miri. Doing crypto algs through miri is so slow

#

And it's using a lot of memory ferrisBorgar

haughty mica Oct 3, 2022, 2:26 PM

#

Very cool

#

If you're willing to share it I can add it to the benchmark suite if it is unique

thorny venture Oct 3, 2022, 2:35 PM

#

https://github.com/conradludgate/delicious/blob/0153130259164014fc2cb84b66b45f96b6eda343/no-way/src/jwa/kma/pbes2_aes_kw.rs#L383-L407

#

It does 4000 iterations of pbkdf2 basically

#

So uhh, it makes sense that it's slow

haughty mica Oct 3, 2022, 2:36 PM

#

Ehhhh

#

There's "Miri is slow" then there's all the issues I've been closing about how Miri is slow

thorny venture Oct 3, 2022, 2:37 PM

#

I'll let it run to completion. in debug the test took 0.13s

#

well, it's at least 3 minutes

#

11 minutes and still going

#

miri sat at 8GB of memory ferrisWeary

haughty mica Oct 3, 2022, 2:51 PM

#

Yes we need better self-profiling support for this

#

Can you drop the workload size here a whole bunch?

#

Oh count

thorny venture Oct 3, 2022, 2:52 PM

#

sure

haughty mica Oct 3, 2022, 2:53 PM

#

Let's try 64

#

The problem is that the Rust-level profiling support that exists doesn't work if you have to kill the interpreter

thorny venture Oct 3, 2022, 2:54 PM

#

15seconds

haughty mica Oct 3, 2022, 2:54 PM

#

typenum

#

Deadge

thorny venture Oct 3, 2022, 2:54 PM

#

lmao ikr

haughty mica Oct 3, 2022, 2:55 PM

#

I don't know where the memory is going though

#

That's a good question

thorny venture Oct 3, 2022, 2:55 PM

#

stream ciphers still can't be represented with const generics ferrisWeary so typenum is necessary

haughty mica Oct 3, 2022, 2:55 PM

#

Do you have any static/once_cell stuff in here?

thorny venture Oct 3, 2022, 2:56 PM

#

not afaik

haughty mica Oct 3, 2022, 2:56 PM

#

I don't even see any big arrays

#

So this is very strange

#

At a guess you have the SRW pileup issue

thorny venture Oct 3, 2022, 2:56 PM

#

It's <linear it seems. I upped the number to 256 and it went to 45s

haughty mica Oct 3, 2022, 2:56 PM

#

That's to be expected nowadays 😉

#

Miri shouldn't be much worse than linear, the big nonlinearity issue was solved

thorny venture Oct 3, 2022, 3:01 PM

#

I guess my 4096 case would take 12minutes, but I think memory was becoming an issue at that point

haughty mica Oct 3, 2022, 3:01 PM

#

I'll look into this later. Probably need to hack up the interpreter to figure out what's going on

thorny venture Oct 3, 2022, 3:01 PM

#

nice

haughty mica Oct 3, 2022, 3:01 PM

#

At a guess, there is a huge borrow stack somewhere

#

The perf probably doesn't go to shit because the lookup cache saves you, but the GC can't fix the memory growth issue

thorny venture Oct 3, 2022, 3:10 PM

#

Yeah I don't see how

    for _ in 1..rounds {
        let mut prfc = prf.clone();
        prfc.update(&salt);
        salt = prfc.finalize_fixed();

        xor(chunk, &salt);
    }

would cause issues ferrisSweat

haughty mica Oct 3, 2022, 3:16 PM

#

You like most Rust programmers are probably not thinking about what your code looks like when compiled without any optimizations

#

In any case, this is a good benchmark because it looks like it stresses the retag_return_place code path a lot, which I know I made slower with the stack cache but I couldn't justify doing fiddly optimization of it then

tender nimbus Oct 3, 2022, 3:20 PM

#

haughty mica You like most Rust programmers are probably not thinking about what your code lo...

And even better, no one except you is thinking about what the borrow stack sizes look like ferrisBut

haughty mica Oct 3, 2022, 3:22 PM

#

Well yeah

#

Partly because it used to be that the whole story was "borrow stacks are always huge"

tender nimbus Oct 4, 2022, 3:38 PM

#

I would go with RustCore of the 3 just to keep the name close to Rust. But I'm infamously bad with names already so :P

@haughty mica "miri-tools" crying on the corner

haughty mica Oct 4, 2022, 4:12 PM

#

Exactly

#

It's now comfortably more than just Miri too

#

I need to properly integrate the ASan option

tender nimbus Oct 5, 2022, 5:05 AM

#

I just had an idea for a way to print clean borrow stacks in Miri (at least with isolation)

Have a flag that sets the "print-alloc-id" and then a function like fn miri_print_stacks() which prints the stacks for the configured alloc id

This would not have tags around for getting the alloc id

You'd have to get the alloc id from a previous run, either through the usual function or just by doing a little ub

haughty mica Oct 5, 2022, 2:50 PM

#

You can already do this btw

#

miri_get_alloc_id just returns a u64, you can print it yourself

#

I dislike this general strategy because it doesn't work if you have nondeterministic behavior, but I agree it works most of the time

#

Also, thoughts on how the GC should interact with this?

#

Should we have another magic function to turn it off temporarily? Warn if it's on and you print stacks?

#

Or run the GC right before each print?

haughty mica Oct 6, 2022, 1:01 AM

#

tough leaf also maybe have the root page be a landing page

root is now a landing page with a search bar. I can't get it to behave on mobile yet, but I'm sure it's just me being bad at webdev.

#

The page is still flipping huge. Might need to address that.

haughty mica Oct 8, 2022, 7:00 PM

#

I'm now pulling --no-default-features and --features from [package.metadata.docs.rs]
I see there's also a thing for rustdoc-args which I am ignoring for now. Seems interesting though.

#

I also have an ANSI-to-HTML renderer which is good enough to handle cargo nextest run so that'll be up soon

#

The downside is that this has exploded runtime because I'm only enforcing a 1-minute per-test timeout

haughty mica Oct 8, 2022, 7:55 PM

#

Any thoughts on what to do if a crate has both metadata.docs.rs and metadata.playground?

#

Do like a... union?

west violet Oct 8, 2022, 11:21 PM

#

Maybe give playground precedence

haughty mica Oct 9, 2022, 12:18 AM

#

L M A O idna has its own test harness

#

RIP

#

Your tests will just never run under Miri

#

Startup time is somewhere beyond 3 hours and 8 GB of memory

haughty mica Oct 9, 2022, 12:19 AM

#

tender nimbus > I would go with RustCore of the 3 just to keep the name close to Rust. But I'm...

You know, it kind if is tools now

#

Maybe that's just the copium kicking in

grim copper Oct 9, 2022, 2:21 AM

#

haughty mica Startup time is somewhere beyond 3 hours and 8 GB of memory

we're at 7.5GB and counting rn

haughty mica Oct 9, 2022, 2:22 AM

#

That code for whatever reason is hitting some code path that defeats the stack cache

#

I'm not certain but it may be a GC interaction

#

Maybe that's just my paranoia

grim copper Oct 9, 2022, 2:23 AM

#

idk, but I'm gonna run it for a few hours to see.

#

I'm only 1 hour in

#

but my desktop is reasonably beefy

tender nimbus Oct 9, 2022, 7:31 AM

#

haughty mica You know, it kind if is _tools_ now

ferrisForgor

haughty mica Oct 10, 2022, 2:06 AM

#

Latest ETA update: 36 days

#

I should probably be more strategic than "rerun every version of every crate with nextest"

tender nimbus Oct 10, 2022, 4:51 AM

#

ferrisConcern

tender nimbus Oct 11, 2022, 8:40 AM

#

https://github.com/rust-lang/rust-clippy/pull/9572

GitHub

Add `as_ptr_cast_mut` lint by Nilstrieb · Pull Request #9572 · rust...

This lint detects calls to a &self-taking as_ptr method, where the result is then immediately cast to a *mut T. Code like this is probably invalid, as that pointer will not have write permissio...

#

Neat

ruby jacinth Oct 11, 2022, 10:14 AM

#

Yay, another lint to ignore everywhere ferrisHmm

wraith wind Oct 11, 2022, 2:34 PM

#

How would you go about fixing https://miri.saethlin.dev/ub?crate=lru&version=0.8.1? It seems to take a pointer into a Box, then insert the pointer and the box itself into a map, moving the box and invalidating the pointer, which is later dereferenced

#Fixing UB in random crates