#Fixing UB in random crates
small apexBOT
tender nimbus
maybe don't make it generic
hey ferris can you godbolt ```rust
use std::mem::MaybeUninit;
struct MaybeUninitConsts<T>(T);
impl<T> MaybeUninitConsts<T> {
const UNINIT: MaybeUninit<T> = MaybeUninit::uninit();
}
pub fn foo() -> [MaybeUninit<u8>; 128] {
[<MaybeUninitConsts<u8>>::UNINIT; 128]
}
example::foo:
mov rax, rdi
ret
hm, nice
hey ferris can you godbolt ```rust
use std::mem::MaybeUninit;
pub fn foo() -> [MaybeUninit<u8>; 128] {
[MaybeUninit::uninit(); 128]
}
example::foo:
mov rax, rdi
ret
hm, nice
not sure whether i trust it 100% though 
#[unstable(feature = "maybe_uninit_uninit_array", issue = "none")]
the fuck core, ???
ah, a tracking issue was added recently
oh god i hate how basically every call to copy or copy_nonoverlapping to the same allocation has SB issues because of course it fucking does, people are reasonable to write code that has these issues
self.buffer.get_unchecked_mut(len)
can we get a clippy lint please 
wait, what is it doing
wtf
this is straight up reading uninit memory
what
ah, it needs to read it as MU, then it's fine
rerunning this beauty #932319394724479037 message
tender nimbus
writing extremely cursed (fish) shell scripts again to sort the ub crates by downloads
curl https://miri.saethlin.dev/ub > miri.html
curl -L https://static.crates.io/db-dump.tar.gz -o dump.tar.gz
tar -xzf dump.tar.gz
cat miri.html | sd '.*crate">(.*?) (.*?)</div><div class="status">UB: (.*?)</div.*' '"$1","$3"' > ub.csv
for c in (cat ./ub.csv) ;
set NAME (echo $c | xsv select 1)
set COUNT (xsv search "^$NAME\$" s_crates.csv | xsv select -n 2 | sed -n '2p')
echo "$COUNT $NAME $UB"
end > downloads.txt
cat downloads.txt | sort -nr > result.txt
haughty mica
They're already sorted by recent downloads
tough leaf
lol
tender nimbus
well, but it doesn't show the download count so my script is still not useless 
tough leaf
🔵 💙 📘
tender nimbus
west phoenix
simdjson needs unsoundness fixed
tender nimbus
simdjson is a bit annoying because it can't really be mirid
haughty mica
How do you know it has a soundness issue @west phoenix ?
tough leaf
How do you know it has a soundness issue <@341486397917626381> ?
i found it handing out non-utf8 Strings
which isn't that bad but it's still unsoundness
still gotta report that
haughty mica
Yeah that's not great
west phoenix
How do you know it has a soundness issue <@341486397917626381> ?
Well, first I saw it allowed clippy::uninit_vec
and that means there is definitely ub somewhere else
west violet
There’s a lotta sketchy stuff in that crate tbh
The nice thing is that it’s got extensive tests & fuzzing so changes are easy
haughty mica
and that means there is definitely ub somewhere else
There's UB right there, if rustc starts adding noundef
west phoenix
the uninit vec stuff was fixed iirc
tender nimbus
Yes, they fixed it shortly after I opened an issue, so they are definitely open for soundness fixes
tough leaf
There's UB right there, if rustc starts adding `noundef`
yeah, we currently do emit noundef for the same types that mem::uninitialized panics for
anything with an invalid value
i don't think noundef does much right now but it would be interesting to see if any benchmarks improve if we emit it everywhere we can
haughty mica
But where can we
Oh fucking hell
pub(crate) fn parse_str_<'invoke>(
input: &'de [u8],
data: &'invoke [u8],
buffer: &'invoke mut [u8],
mut idx: usize,
) -> Result<&'de str> {
use ErrorType::{InvalidEscape, InvalidUnicodeCodepoint};
let input: &mut [u8] = unsafe { std::mem::transmute(input) };
lmaooooo
#[allow(
clippy::if_not_else,
mutable_transmutes,
clippy::transmute_ptr_to_ptr,
clippy::too_many_lines,
clippy::cast_ptr_alignment,
clippy::cast_possible_wrap,
clippy::if_not_else,
clippy::too_many_lines
)]
mutable_transmutes is just another annoying lint just like the rest right
haughty mica
mutable_transmutes is literally just a UB detecting lint
west phoenix
btw this crate is used as an optional dep of serenity (the discord bot library), so like, is probably being used in production rn
haughty mica
serenity != production
west phoenix
probably
and by probably, the only reason I'm not running it on my 36k server bot is because it had some deserialization issues (before I found out about this)
haughty mica
Also in general I'm not worried about these sorts of issues "in production", I'm worried about them going forward, or how they reflect on Rust overall
What do you mean by "issues"?
west phoenix
haughty mica
Oh, snowflakes
I don't remember exactly but those do tickle some uncommon scenario for deserializers
west violet
```rust
pub(crate) fn parse_str_<'invoke>(
input: &'de [u8],
...
Oh god it actually copies into input https://github.com/simd-lite/simd-json/blob/ae48dc4ee34c8cc698c73ac599bcfdd648a53cd4/src/avx2/deser.rs#L144-L146
haughty mica
And it uses clone_from_slice for &[u8] which is not awful but just why
west violet
So they're aware but don't care?
tough leaf
yes
well
they might not know it's UB
https://docs.rs/simd-json/latest/simd_json/serde/fn.from_str.html
parses a str using a serde deserializer. note that the slice will be rewritten in the process and might not remain a valid utf8 string in its entirety.
tough leaf
agreed
ruby jacinth
If input is a static string literal wont writing into it crash instantly
haughty mica
I suspect that scenario is avoided somehow
tender nimbus
```rust
pub(crate) fn parse_str_<'invoke>(
input: &'de [u8],
...
this is what happens when you port a c++ library 
grim copper
It’s sad that it’s so cursed, lemire’s libraries deserve better than a crappy port
tender nimbus
https://github.com/ParkMyCar/compact_str/pull/100#pullrequestreview-1002178051
I spent most of the time fixing the arc module
knotty oar
So they're aware but don't care?
i know the author personally, and i wouldn't be surprised if this is true
haughty mica
once_cell now does pointer stuffing with as-casts 😩
west violet
Is that good or bad
haughty mica
Bad, because it's such a core library that this update will cause a lot of things to die under -Zmiri-tag-raw-pointers
west violet
Gotcha
(I didn't know if this was a "yay, it no longer uses transmute!" or something)
haughty mica
Ah. I've yet to run into a crate that does pointer transmutes which I can't replace with wrapping operations. I'm sure they exist, but haven't hit one yet.
Miri thinks one of the once_cell tests deadlocks
That is probably not great
haughty mica
Oh Truuuuuu
ruby jacinth
better than makefiles at least
ruby jacinth
eh they're pretty bad
extremely messy when they get big, and it's really easy to write them in a platform dependent way
haughty mica
Don't get me wrong, Cargo is so much better. I've just seen too many "Makefiles but with my personal nits fixed" things
tender nimbus
these are the sort of -Zmiri-measureme that i love
that's a lot of time spent offsetting pointers
these zero cost abstractions become very much not zero cost under miri
haughty mica
What crate is this
tender nimbus
compact_str
there's probably a huge string in the test suite, I didn't take a closer look
that's wild
tawny coyote
I dont know shit about provenance and stuff, so noob question here:
ptr-int transmute
int-to-ptr cast
Does that mean transmute::<*const T, usize> is UB, but usize as *const T is not?
What about the other other directions, transmute from int to ptr and cast from ptr to int?
proper belfry
I dont know shit about provenance and stuff, so noob question here:
> ptr-int t...
cast from ptr to int is definitely OK, it will probably be deprecated though
tender nimbus
transmuting pointers to integers has been made ok recently
proper belfry
int to ptr transmute I think is okay, it probably just creates a pointer with zero provenance
proper belfry
transmuting pointers to integers has been made ok recently
oh that’s nice. so it has the semantics of .addr()?
tender nimbus
casting integers to pointers is complicated
tender nimbus
oh that’s nice. so it has the semantics of `.addr()`?
yep, if you look at the implementation of .addr, it's now a transmute
proper belfry
nice, this makes a lot of sense!
tawny coyote
what makes transmute there better than as usize?
proper belfry
what makes transmute there better than `as usize`?
It doesn’t expose the address of the pointer, theoretically allowing more aggressive compiler optimizations
tender nimbus
oversimplified explanation
provenance is an extra made up part of pointers that control what permissions they have to access
if you cast an integer to a pointer, what provenance does it get?
it looks whether a provenance has been "exposed" for this address, if yes, it gets it, if no, the pointer will have no provenance and can therefore not be dereferenced
to expose a provenance, cast a pointer to an integer using as
the transmute doesn't expose this
this is all used for compiler optimizations, so the transmute could (in the future) get slightly better compiler optimizations
pastel lily
transmute is claiming “this pointer is just an integer” where as (or other methods) tell the compiler you want to actually do things on pointers
tender nimbus
transmute is "just give me the address btw"
as is "give me the address and make sure that I can cast this back to a pointer later"
tawny coyote
ah i think get it, also just noticed there's an explanation of provenance in the docs now
haughty mica
transmuting pointers to integers has been made ok recently
No I think it was made not okay at all
pastel lily
A pointer is an address plus some abstract machine state that matters to the compiler and may or may not exist at runtime (on platforms like CHERI). So while the address part can always fit in a usize, you have to do Other Things to operate on that other state.
tender nimbus
No I think it was made not okay at all
haughty mica
Oh that's interesting
tender nimbus
as you can see @tawny coyote, these kinds of rules are still in progress 
tawny coyote
as you can see <@331386192928964610>, these kinds of rules are still in progress...
yep but very interesting, I'll read through some of the stuff
tender nimbus
before this comment, transmuting a pointer to an integer was considered ub
tender nimbus
yep but very interesting, I'll read through some of the stuff
if you want a better explanation of all this, read through ralfs blog posts
haughty mica
The heart of the problem is that observing the address of a pointer has implications for what optimizations you're allowed to do
But to be honest all this stuff is in the weeds, because stacked borrows with untagged still doesn't support noalias, and I'm waiting for stacked borrows with wildcard to land before I ask Ralf if that supports noalias
pastel lily
C lets you do whatever you want with pointers and that inhibits so many optimizations
tender nimbus
C lets you do whatever you want with pointers and that inhibits *so many optimiz...
well no, not really
tawny coyote
ima just read https://plv.mpi-sws.org/rustbelt/stacked-borrows/paper.pdf for maximum coolness
haughty mica
Well
tender nimbus
good luck, though I doubt that you'll understand much (i don't either )
haughty mica
The problem with the C rules is that it's not clear how restrict is valid
pastel lily
Well every pointer is basically exposed and you can do things like serialize them, isn’t it
tender nimbus
nah
pastel lily
and nobody uses restrict even if that helps some
tender nimbus
you have to expose them manually still
under PNVI-ae-udi
which is what c will probably get
pastel lily
I’ve only ever seen restrict used in libc
tender nimbus
funnily c provenance is still not really settled as well
haughty mica
I’ve only ever seen restrict used in libc
It's used here and there in libraries
pastel lily
And I’m sure that nobody calls it right
tender nimbus
and it's used in all good cuda code, so I've heard from our local cuda wizard
c still has provenance, you're not allowed to go out of bounds with pointers, still have to expose them for int2ptr casts, so c pointers are limited
but not as limited as rust pointers
haughty mica
Probably
It's still possible for the the compiler/lang teams to just say "oh dear we can't break anyone's code!" and simply remove all noalias from the compiler and let everyone live with the regression.
tender nimbus
haughty mica
Wait a second, Mr. Weak Memory Effects pasted 2 backtraces into std::sync::mpsc
This is bizarre
tender nimbus
now, i wouldn't be surprised if std::sync::mpsc was deadlocking
tf, running just the deadlocking test does not make it deadlock
haughty mica
Anyway
tender nimbus
but it's definitely this test, ignoring it makes it not deadlock
west violet
So if transmuting ints is chill can bytemuck add them back?
tender nimbus
i don't think the transmute in the other direction is cool
tender nimbus
i wonder why the stampede_once is disabled under miri
ah, i guess the old miri scheduler didn't support it
it works now
it doesn't deadlock with all miri seeds, but it happens to deadlock you run the entire test suite with seed 0
the miri seed 🅱️ reproduces the issue with just the one test
the test is doing some funky things with channels, threads and oncecells
it does look good to my brain, though i don't think my brain is very good at this
hmmmmmmmmmm, this is interesting
if i add a SeqCst fence to the initializer it looks great ("looks great" meaning that it passes 0x11 seeds and then fails because, uhm, deallocating while item is protected: [SharedReadWrite for <222825> (call 65038)] (somewhere in mpsc )
the fun thing is that there's a nice comment on initialize
Safety: synchronizes with store to value via SeqCst read from state,
yet the inner method only usesAcquires
haughty mica
Ralf thinks this might also be a futex issue
Comment in the zulip
I have not seen that protector error, but there is an issue with dangling Arc
haughty mica
I coaxed a SIGSEGV out of another Facebook codebase ayyyyyyy
haughty mica
And a Solana crate lmaooo
west violet
Time to crash the blockchain
haughty mica
Oh boy oh boy -Zrandomize-layout + -Zbuild-std is starting to turn up crates that SIGILL
That's almost certainly misuse of an unsafe API in the standard library
I already see one crate we use at work which is just so exciting
In case anyone wants to take a crack at some of these in the meantime:
abomonation/0.7.3
plotters-bitmap/0.3.1
encode_unicode/0.3.6
safe-transmute/0.11.2
fallible_collections/0.4.4
plotters/0.3.1
heapless/0.7.13
wasmer/2.3.0
typed-index-collections/3.0.3
slice-deque/0.3.0
swc_ecma_transforms_compat/0.102.0
swc_ecma_transforms_optimization/0.128.0
parquet/15.0.0
I already know about the issues in abomonation, heapless, and safe-transmute. The others don't jump out at me as familiar
grim copper
parquet is apache so hopefully they should be able to fix issues well enough
also it underpins arrow / datafusion / polars
tender nimbus
speedy web segfault
ruby jacinth
Web3isgoinggreat
tender nimbus
swc isn't web3 crap, it's the web compiler/bundler for web2 things
knotty oar
web(-3.0)
tender nimbus
so, I'd actually care about swc
grim copper
also it's a typescript compiler
tender nimbus
with no typechecking 
grim copper
with no typechecking <:ferrisPensive:857440844549324851>
hah didn't know that. I guess it kinda makes sense is compile speed is a priority though
I guess your language server can do the type checking instead..
haughty mica
parquet is apache so hopefully they should be able to fix issues well enough
The parquet crate has a lot of soundness problems historically
I mean like users opening tickets about segfaults
haughty mica
In the case of parquet it seems kind of like they didn't care at first
They do now and one developer is trying to reimplement the whole thing
haughty mica
Creating a mutable reference or doing write through a raw pointer removes all tags for the memory in question that post-date the source of the reborrow or the pointer for the access
This is not a good behavior in SB
grim copper
what does SB stand for?
oh stacked borrows
nvm
I wonder if we could get a mod to pin https://miri.saethlin.dev/ in this channel - it's a super cool tool!
haughty mica
It's in the opening comment I think
tender nimbus
yes
grim copper
can you get to the opening comment without scrolling way up?
ruby jacinth
Just ask mod to pin it
grim copper
@tender nimbus are you able to pin here?
tender nimbus
<@&631915156854538260> can you pin the original post
this is also something for forum feedback
wintry forge
tender nimbus
thanks
grim copper
cheers m8
wintry forge
grim copper
I wonder how many of these errors still apply if you ignore provenance
looks like even stuff like once_cell is failing due to provenance checks
tender nimbus
yeah, most of these are provenance related
There are also tons of nullptr derer because bindgen tests
haughty mica
Not very many
tender nimbus
738
haughty mica
I should upload the version of this with SB disabled
tender nimbus
yeah that would be very useful as well
not seeing so many uninit memory (http) is soo nice 
grim copper
is SB == provenance stuff? I always kinda thought that provenance is some subset of SB?
tender nimbus
sb is a model for handling provenance basically
the concept of provenance itself is not rust specific (also happens in every other compiled language with optimizing compilers)
rerunning my script on the httpless data reveals the new top ub causer
grim copper
yeah, I understand that much, I just think I wasn't understanding the relationship, but that makes sense
tender nimbus
bytes already has fix they just need a new version release 
tender nimbus
🚀p🚀e🚀r🚀f🚀o🚀r🚀m🚀a🚀n🚀c🚀e🚀
pastel lily
perf (but we didn't profile)
grim copper
wait... is the rust-crypto crate different from the rust crypto org?
it is isn't it...
tender nimbus
https://github.com/DaGenix/rust-crypto/ not in the org
grim copper
ffs
tender nimbus
no, zero init
grim copper
oh right it's a u32
proper belfry
last updated 2016
don’t think there’s merging that
tender nimbus
oh
proper belfry
grim copper
oof
proper belfry
tender nimbus
i think you're better off PRing these people https://crates.io/crates/rust-crypto/reverse_dependencies
grim copper
most downloaded rev dependency last updates 2 years ago
of course it's a merkle tree crate and is definitely in every rust blockchain project
💯
tender nimbus
merkletree had its last update a month ago
grim copper
oh I guess only last release 2 years ago
grim copper
rerunning my script on the httpless data reveals the new top ub causer
what is the number on the left?
tender nimbus
how many crates failed miri because they had the bad dep on the right
grim copper
ah I see
IMO the most useful metric would be sorting unsound crates by recent downloads
tender nimbus
they are already sorted by downloads on the website
idk whether it's total or recent though
grim copper
oh, didn't know that
tender nimbus
writing extremely cursed (fish) shell scripts again to sort the ub crates by dow...
you're not the only one
tender nimbus
In case anyone wants to take a crack at some of these in the meantime:
```
abomo...
checking out the swcs
knotty oar
relax i'll handle the prs if you don't want to
tender nimbus
~~relax i'll handle the prs if you don't want to~~
can you do rollups for edits of my forum post
nice, i got the sigill as well
knotty oar
what forum post 😛
grim copper
every time I see a rollup I think of these
tender nimbus
what forum post 😛
the op of this thread
knotty oar
linkez-moi
grim copper
tis pinned
tender nimbus
fun, it sigills somwhere in slice indexing
uuuhm, why does it hit a ud2 instruction after a ret
ah because a branch nvm
knotty oar
ah wait you mean saethlin's repo
tender nimbus
i mean this
knotty oar
ye that
pastel lily
tender nimbus
the goods news
a bunch of tests pass now
the bad news
it still sigills at some point
it's really nice that cargo tells you the name of the test binary
ah, the same pattern 20 lines above
so lmao, it's not even the randomize-layout that's going on here but the debug asserts from build-std
and it passes!
tender nimbus
ah yes, love it when ci does that
as you can see, im a very busy person on github with so many notifications
knotty oar
ah github notifications the thing that nobody checks
haughty mica
I obsessively check my notifications
It's the only way to stay on top of like 10 PRs
tender nimbus
same
knotty oar
i check it via my email
haughty mica
To each their own. My email is far too messy for that
knotty oar
ya fair
i wouldn't mind using github notifications, i thought it would solve my problems but ugh the UI just makes me not remember it exists
west violet
Oh boy oh boy `-Zrandomize-layout` + `-Zbuild-std` is starting to turn up crates...
good, good
Good that it faults abomination, nice to have confirmation it’s working properly
west violet
I still have no idea how to fix the padding thing
No one’s given me any help I know how to act on
west violet
My extension of randomize that adds random padding before fields
There’s a bug with option on guaranteed niche types rn
But I don’t know how to express "if is option with guaranteed niche T"
west violet
Well that’s the thing, it’s just option specifically
haughty mica
I think you're making this too hard for yourself
west violet
And I’m not at all familiar with the compiler so I don’t know anything
The biggest issue is that I just don’t know what’s going on
haughty mica
You could just skip everything that looks like Option.
I for one just stomp around the docs until I find something that looks useful
For example, if I search for layout there are a few helpful looking functions, depending on what you have https://doc.rust-lang.org/stable/nightly-rustc/rustc_middle/?search=layout
haughty mica
Perhaps
west violet
Hum, I don’t think so
Option is just repr rust
I’m also not entirely sure what my function "knows" about
I don’t know if it knows it’s within an option or anything since it’s just calculating an aggregate type layout
tender nimbus
some of those actually just build-std debug assertions instead of randomize layout (swc was just a debug assertion)
tender nimbus
But I don’t know how to express "if is option with guaranteed niche T"
I would just express "if it's option, don't"
Even if there is no niche
west violet
Eh, that's sub-optimal
haughty mica
I suspect most of them are just from turning on the stdlib debug assertions
Though it's possible that adding randomize-layout in there produced some problems which are only detected by the debug assertions
west violet
Also potentially true, you could check with build-std without randomize
haughty mica
It's worth noting that the only other way I have to detect layout problems is looking for a SIGSEGV from a randomize-layout run, which basically only happens when you confuse pointers
Looking for test failures is too hard
I plan on diagnosing all of these crashes individually, so while it might be interesting to do build-std without randomize I think I'll power through
west violet
This is the function I'm working in btw if you have any ideas https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/layout/struct.LayoutCx.html#method.univariant_uninterned
I'm just confused tbh
This is a cry for help
haughty mica
Ask more detailed follow-up questions on the zulip
People there are trying to be helpful but they probably are assuming you know more than you do
You're also not in compiler/help
west violet
Ah whoops
haughty mica
They probably don't mind much
haughty mica
You could ask someone to move it for you
But also if someone hasn't suggested it already they probably don't care that much
It sure is cool that the authors of cryptography crates test their code with sanitizers and/or Miri
west violet
Sarcasm, I assume?
haughty mica
It's a brown M&M angle
west violet
I'm not familiar with that reference
haughty mica
"If I came backstage, having been one of the architects of this lighting and staging design, and I saw brown M&Ms on the catering table, then I guarantee the promoter had not read the contract rider, and we would have to do a serious line check" of the entire stage setup, Roth said.
west violet
Ahhh, smart approach
Yah, I guess that's fairly indicative of them caring about safety and whatnot
haughty mica
Yes this is a harmless little UB in your test, but the fact that it's in here tells me that nobody is using ASan on this library
tender nimbus
why would one even use ASan or Miri
my code is perfect, i don't need such commonfolk tools
haughty mica
How do I put up an advisory that just says "holy shit do not use this crate please"
west violet
Abomonation be like
haughty mica
Whiplash from
- Use after free
- Protector error due to inserting into a hashmap while holding a reference across the insert
west violet
Starters I guess?
I donno the term
I want to run asan on github ci, do you have any examples of that
haughty mica
Normal cargo test but with
env:
RUSTFLAGS: -Zsanitizer=address
Wait really?
haughty mica
Yes that's all you have to do
west violet
You don't need to run it under anything special?
haughty mica
Nope, just a nightly toolchain and a flag
haughty mica
ASan supports MacOS and on Windows it's supported by clang
You can mix ASan and UBsan, which is not supported on Rust because there's no reason to
But other than that no you cannot mix them, you need to do them one at a time because their shadow memory runtimes collide
west violet
Gotcha
tender nimbus
Do you have any sanitizer ci bases for me btw?
arc_swap has great CI
tender nimbus
How do I put up an advisory that just says "holy shit do not use this crate plea...
make a 9.8 critical cve , just like the one from failure
haughty mica
Holy shit it got a 9.8 for being EoL
Oh it's type confusion. That's still probably not a 9.8 but it's not just for being unmaintained
tender nimbus
it got a 9.8 for "if the programmer willingly exploits a weakness in a library they are using, they can cause ub in safe code!"
which is very yikes
haughty mica
To be fair, rustsec is part of the problem here
proper belfry
it got a 9.8 for "if the programmer willingly exploits a weakness in a library t...
can we give every C library ever a 9.8 CVE
haughty mica
The fact that rustsec keeps saying "attack vector: network" is probably a big part of the problem here
tender nimbus
attack vector: funny developer
i will suggest an improvement to the failure advistory
i mean
tender nimbus
i have no idea what to even fill out here
knotty oar
How do I put up an advisory that just says "holy shit do not use this crate plea...
report it to wg-sec they should handle it 😛
tender nimbus
like, there are no attack vectors
haughty mica
Attack vector: Network + Confidentiality: High means you can use this vuln to dump the whole contents of a web server
knotty oar
i have no idea what to even fill out here
eenie meenie minie moe
tender nimbus
and if a developer willingly writes bad code
then everything would possibly be the vector
haughty mica
report it to wg-sec they should handle it 😛
every single CVSS score they put out is like this
haughty mica
Like, look at this: https://rustsec.org/advisories/RUSTSEC-2019-0035.html
knotty oar
filling them out realistically gave me this
congrats nils, you are now a member of wg-sec
haughty mica
Unaligned access means you can... totally own a server over the network apparently?
tender nimbus
~~congrats nils, you are now a member of wg-sec~~
inb4 updating the log4shell advisory to do
tender nimbus
Unaligned access means you can... totally own a server over the network apparent...
yeah see, if the memory is not aligned as you want, it could rebel! and if memory rises up against you, everything is bad
haughty mica
FYI there are 3 log4shell CVEs, people lost their minds over all 3, and 2 of them are rated lower than most of the RustSec CVSS scores
knotty oar
aye
tender nimbus
see, the log4shell attack are only hypothetical
who in their right might would put up a java webserver????
yet when I grep through my code for __private_get_type_id__ , I get hundreds of results
this is critical to web integrity
i'm giving it low here because idk, when a developer fucks this up maybe they could get hacked
why is there no attack vector "access to source code"
Oh right, because then the CVE would make no sense!
haughty mica
as_mut_ptr considered harmful: Now without Stacked Borrows
tender nimbus
tender nimbus
`as_mut_ptr` considered harmful: Now without Stacked Borrows
oh no what happened
haughty mica
Two crates doing a let buffer = Thing::new().as_mut_ptr();
haughty mica
Ah found another crate that does the same
tender nimbus
is this a clippy lint
not like I expect people that don't run Miri to run clippy
But one can hope
haughty mica
We should really have a lint against this
tame jewel
`Attack vector: Network + Confidentiality: High` means you can use this vuln to ...
No idea how 'tokio has a race condition' gets 8.1 while unaligned memory access gets >9.x
With no evidence that this even leads to practical miscompilation given anywhere. (there's double frees with less score)
Incorrect hash in sha2 gets 9.8 by means of 'Availability: High', 'Confidentiatlity: High'.
I have yet to discover how to reveal information or crash anything with this when it swapped two blocks of data during hashing.
CVE for libraries is entirely political, change my mind
haughty mica
I agree, libraries should probably not get CVEs
I think I could make an argument that all my unsound advisories can be prompted to a 9.8 CSS CVE by their logic but I don't have the patience for that
west violet
Do you know why miri wouldn't recognize -Z miri-strict-provenance?
- name: Run miri
uses: actions-rs/cargo@v1
env:
OS: ${{ matrix.os }}
PROPTEST_CASES: "10"
MIRIFLAGS: "-Z miri-strict-provenance -Z miri-check-number-validity"
with:
command: miri
args: test --all-features
error: unknown debugging option: `miri-strict-provenance`
Error: unknown debugging option: `miri-strict-provenance`
error: test failed, to rerun pass '--lib'
Error: The process '/home/runner/.cargo/bin/cargo' failed with exit code 1
i think you need to write it as -Zmiri-thing
miriflags parsing 
i think it just does a split space
west violet
Lovely
haughty mica
It's awful
west violet
What's the isolation one, Zmiri-disable-isolation?
haughty mica
Yes
tender nimbus
yes
haughty mica
They're all in the readme
tame jewel
Lol, there's literal XSS vuln in the database with lower scores on confidentiality than the above race conditions. smh.
tender nimbus
you're making fun of the race condition
but it gets even worse
a soundness issue (realistically non-issue) got a 9.8
tame jewel
So any as_ptr() as *mut T will get the same, then?
tender nimbus
So any `as_ptr() as *mut T` will get the same, then?
i mean, that's an actual stacked borrows issue (but doesn't deserve a cve of course)
what I'm talking about is
"a macro assumed that you wouldn't create a function with a very special name, and if you did create that function (you have to do this on purpose) then you can cause ub in safe rust"
tame jewel
Nvm that it's an actual sb issue. We already found it can't escape the local analysis in llvm<14, so practically it can't cause miscompilation.
haughty mica
Casting pointers around is always valid. Are you referring to doing a write through that pointer?
tame jewel
referring to returning this pointer via a library interface as &mut _
And in |x: &mut [u8]| &mut *(x[..4].as_ptr() as *mut [u8; 4])
which is UB according to SB but the llvm compilation will never know that the pointer can't be writtent through
tame jewel
That's what I'm trying to say, I have no idea how to handle those advisories for that reason.
tender nimbus
only things that can actually break should be an advisory
tame jewel
complexity: Low, impact: None?
tender nimbus
isn't complexity about attack complexity?
tame jewel
the difficult thing is that future Rust compiler can make it break, and would then have to bump up all the issues
even compiling the same thing with gcc might do it, I don't know
There's no way to express that dependency in cve, because it's not built for libraries
haughty mica
I'm not sure they even deserve complexity: low
It's more like complexity: unknown
The exactly miscompilation that may arise isn't known, and even if one does occur I have no idea what the odds are that it would even get deployed
And what's the user interaction?
tender nimbus
these attack vectors really really aren't made for these kinds of things
like, not at all
how the fuck should i know whether my library ub is exploitable over the network or not
haughty mica
Correct, they're designed for applications not libraries
Another one that I cannot patch
pastel lily
Is this self referential too
haughty mica
Yes, but the stack use after return is much worse
The self referential stuff could be patched over with some AliasiableBox or whatnot
Also the GitHub repo doesn't exit anymore so this one too will not be patched
tame jewel
Yikes, is that meant to be some form of sentry?
haughty mica
What do you mean by that?
tame jewel
Is that a single linked list, and it tries to use out as a form of termination?
west violet
Apparently mpsc has a double free?
haughty mica
That's not a double free
west violet
Oh yah, a "protected deallocation"
haughty mica
Protectors are a direct expression of the dereferenceable LLVM attribute that goes on references
west violet
So why would this be happening?
haughty mica
Does this go away if you pass -Zmiri-disable-weak-memory-emulation
west violet
Related, I thought miri supported windows threads?
haughty mica
It's possible that it doesn't. All I know is the Windows support is very flaky
west violet
Darn
haughty mica
The problem with Windows is that there are precious few contributors for it
tender nimbus
Related, I thought miri supported windows threads?
not yet
but @golden summit is implementing them right now
west violet
Does this go away if you pass `-Zmiri-disable-weak-memory-emulation`
No it does not
This is happening on mac btw
haughty mica
That's very exciting
west violet
We'll see if it happens on linux, I assume not
lmao
warning: associated function is never used: `name_cstr`
--> /Users/runner/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/sys/unix/fs.rs:763:8
|
763| fn name_cstr(&self) -> &CStr {
| ^^^^^^^^^
|
= note: `#[warn(dead_code)]` on by default
warning: `std` (lib) generated 1 warning
wat
west violet
Oh yah, is -Zmiri-symbolic-alignment-check good though?
haughty mica
Define good
west violet
Should I use it
haughty mica
It has false positives, so I wouldn't suggest it normally. You need to manually diagnose each thing it finds, or run Miri a bunch of times with different -Zmiri-seed values
west violet
Alright cool
haughty mica
It tends to not find false positives because most people do not manually align things, but when they do
west violet
What flags would you recommend for CI then I guess is a better question
haughty mica
-Zmiri-tag-raw-pointers -Zmiri-disable-isolation
tender nimbus
how does symbolic-alignment-check work, what does it do?
west violet
It checks alignment symbolically 
That is unironically how it works though
You can deduce a lot of things from symbolic execution
Basically, even though you don't know the exact pointer values, you know some attributes it has and the invariants of the operations performed on it
west violet
Like, if I have <ptr align(16)> and I perform <ptr align(16)> + 16 I know that it's still aligned since forall { P: Ptr }, P % 16 == 0, (P + 16) % 16 == 0
tender nimbus
oh god, mockall is super cursed
haughty mica
What it says is "a pointer to a u8 can never be used for a read which required an alignment greater than 1"
west violet
Oh
haughty mica
Frankly it's amazing that this doesn't just hork on everything
If you inspect the address of a pointer to walk it up to a correct alignment, it has no idea and you get a false positive
west violet
Although I guess technically it could actually use concolic execution here
haughty mica
Yes it could and that would be awesome
Misaligned pointers are very common in the Rust ecosystem and they're a pain to debug because Miri only catches them sometimes, and Ralf has a theoretical concern about a better alignment strategy for what people usually do
The official solution is "run Miri a bunch with different seeds"
tender nimbus
yeah alignment is hard because you can hardly detect randomly getting correct alignment
west violet
@neon tiger implement concolic pointer alignment checks in miri
haughty mica
Honestly I think just maximally misaligning allocations would be way better
haughty mica
It's a very easy implementation and anecdotally it doesn't suffer the concern Ralf has
tender nimbus
Honestly I think just maximally misaligning allocations would be way better
that does sound like the best solution
west violet
Robust and accurate checks and perturbations give the best results
haughty mica
I need to bug libs about doing that in the standard library
tender nimbus
today i wrote another very cursed shell script
whether a list of repos contains ub
west violet
Oh no it do be broken on linux
That's with -Zmiri-tag-raw-pointers -Zmiri-disable-isolation
tender nimbus
unsafe fn tm_array<T, U, const N: usize>(array: [T; N]) -> [U; N] {
let array = ManuallyDrop::new(array);
unsafe { array.as_ptr().cast::<[U; N]>().read() }
}
let uninit = MaybeUninit::<[T; N]>::uninit();
let uninit = unsafe { tm_array::<T, MaybeUninit<T>, N>(MaybeUninit::assume_init(uninit)) };
wtf is this doing
neon tiger
<@320000638329290752> implement concolic pointer alignment checks in miri
oof
I was looking at the weak memory stuff and thinking "wow it's so cool that we can do this now but kinda sad that it's random"
haughty mica
```rust
unsafe fn tm_array<T, U, const N: usize>(array: [T; N]) -> [U; N] {
...
That's a transmute
Without the size check
pastel lily
does transmute not like const generics?
haughty mica
Transmute does not like generics
west violet
Without the size check
It's faster 😎
pastel lily
yea
haughty mica
I don't grasp the code at the bottom
tender nimbus
it's trying really hard to do ```rust
let uninit: [MU<T>; N] = unsafe { MaybeUninit::uninit().assume_init() };
OH
tender nimbus
(but accidentally makes an uninit t)
haughty mica
Nice
pastel lily
fun
haughty mica
I think the top cause of use after free is people trying to test their "clears the memory on drop" things
haughty mica
Not it it's a type that owns the data
I fixed the one in ed25519-dalek because it's non-owning
pastel lily
ah
haughty mica
The right way to test them is probably with a custom allocator actually
pastel lily
ah yea
haughty mica
let ptr: *const u8 = mem::transmute(&self.to_be());
You do not love to see it
pastel lily
what the
thorny venture
i hope you're proud
:)
pastel lily
but why
pastel lily
my unsafe code typechecks so it must be right
tender nimbus
pro tip: if code doesn't type check, insert transmute or transmute_copy to fix it
thorny venture
it's trying really hard to do ```rust
let uninit: [MU<T>; N] = unsafe { MaybeUni...
Tell std to stabilise the MU APIs already for arrays
I'm tired of always rewriting them
pastel lily
same
neon tiger
same
tender nimbus
uhm
haughty mica
This person tried to hack around a lifetime error and wrote a use after free instead 
west violet
I'm having a miri moment
tender nimbus
west violet
It's been running for 40 minutes
west violet
Doesn't that kinda undermine the checking
haughty mica
Yes but also if this never finishes or OOMs the machine it's on that also undermines the checking
west violet
Fair
tender nimbus
otherwise, if you have a few tests that take very long but most tests don't, cfg(miri)ing them out is also fine
haughty mica
Do what you can on this test case and head on to other code is how I think of it
west violet
Ah yep, 2048 iterations with 32 threads
tender nimbus
i understand why miri doesn't like it
get that down to a lot less iterations and threads
haughty mica
Don't shrink the threads
Also what code are you running I want to profile it
If you use less threads it's harder to hit some race conditions
golden summit
hmm I wonder if MIRI would get noticibly faster if you threw it into souper
haughty mica
No
I mean yes, but not a lot
I have a PR that takes the edge off of SB but it's not a fix
golden summit
I guess it's mainly from the algorithm yeah
west violet
haughty mica
No it's mainly because the implementation leaks tags
west violet
It seems to be hit-or-miss though, probably seed based
haughty mica
I just want to watch it run
The problem with SB is that its design mandates a linear search, and because the runtime doesn't know when a pointer goes away, those linear searches grow in size over the course of program execution
For small programs you don't notice SB overhead hardly at all, then for large programs it eventually eats all your memory
west violet
Can't you at some points stop searching?
Like, once you hit your required access you can stop for something like a read, right?
haughty mica
Yes, that's what it does
You search top down and you add to the top, so if you're constantly searching for the same pointer but also creating new pointers, because for example you're reborrowing... yeah.
To be clear, you don't search for acccess in SB you search for tags
west violet
I was meaning stuff like SharedRead or whatever
west violet
Store the tags in a vec and simd search them
haughty mica
That does not fix the algorithmic problem
west violet
This is true, but isn't the linear nature fundamental?
haughty mica
I have a PR that does some clever caching and it neutralizes the SB overhead for some programs
The linear nature is fundamental, but the leak isn't. Oli and I think there is some way to implement a garbage collector for tags
west violet
Ahhh, gotcha
tender nimbus
@haughty mica will you publish the disable-stacked-borrows run on your website?
tender nimbus
west violet
I still vote for a volunteer-based thingy
I'd totally run a container for you, I'm sure others would as well
haughty mica
Yeah on one core though?
Really what I should do is physically move my computer farther from my bed so I can run this while I'm asleep
You know if people want to help, figuring out some kind of caching scheme for build artifacts that doesn't also let crates corrupt it when I run their tests would be a big help to the runtime speed
west violet
Containers can have more than one core
west violet
You know if people want to help, figuring out some kind of caching scheme for bu...
sccache?
haughty mica
I tried it and I couldn't observe a significant improvement. Probably ~64 processes banging away on one directory that's mounted into 64 different Docker containers isn't fast
west violet
Gotcha
iirc it has s3 support, you could hook it up to mimio (fake s3, we use it in docs.rs for local tests)
haughty mica
You know what, the other thing I should do is actually profile this
west violet
Or maybe do cross-crate scheduling by yourself
Gather all (or some) of the crates that you need tested and build each of their dependency graphs
Merge all the graphs together and then you have a build plan
haughty mica
Yeah see that seems complicated
west violet
This is true
haughty mica
I would rather have something that's slow and not buggy than something that's really complicated but slightly faster
west violet
Fair
I mean it's theoretically the best you can do
Minimal work done, maximal allowed concurrency
tender nimbus
Would be nice if there was a way to have the most common dependencies precompiled
west violet
Wait isn't that called a workspace
haughty mica
Lol like what the playground does
west violet
Wait isn't that called a workspace
Cargo abomination incoming 
Clone them all into one repo and make them one big 'ole workspace
cargo miri test --all
tender nimbus
west violet
Ah that's true
tender nimbus
isn't that basically a shared cargo target dir
true, this might be a good idea
west violet
isn't that basically a shared cargo target dir
I think it's better, cargo may combine their build plans into one big boi
haughty mica
Also some crates stomp around on the filesystem so I'm cautious of just merging them
tender nimbus
On the other hand, are shared target dirs safe for multiple compilations in parallel
west violet
No
haughty mica
Yes
west violet
Really?
haughty mica
You will simply not get any parallelism
west violet
Doesn't cargo take out a lockfile
tough leaf
there's a lock
west violet
Touche'
tender nimbus
that's not great
tough leaf
how often are you really compiling 2 different projects at once
tender nimbus
how often are you really compiling 2 different projects at once
Im not, but miri-tools is
tender nimbus
haughty mica
This <https://github.com/vmware/database-stream-processor/blob/0a58584ad58d1de8b...
I can't see SB craziness in my profile of this, and it eventually dies on some classic as_mut_ptr invalidation
Might just be general interpreter slow
west violet
Miri is still dying with the changes if you want to look at it again ig https://github.com/vmware/database-stream-processor/actions/runs/2481434847
haughty mica
Yeah uh all I see is a gray screen waiting for output
I'm curious to know what it's running right now but I cannot tell
haughty mica
<@176135688666742784> will you publish the disable-stacked-borrows run on your w...
Only has the top 10,000 but go nuts https://miri.saethlin.dev/no-sb/ub
haughty mica
#3 0x00005567c2e03e92 in core::slice::raw::from_raw_parts<u8> (data=0x0, len=0) at src/slice/raw.rs:93
#4 0x00005567c2c1cb4d in font_kit::loaders::freetype::Font::rasterize_glyph (self=0x7ffd363d3c48, canvas=0x7ffd363cfcf8,
glyph_id=3, point_size=9.67741966, transform=..., hinting_options=...,
// Safety:
// we just allocated enough capacity and data_len is correct.
unsafe { escape_field(bytes, self.quote_char, &mut self.data[data_len..]) }
I love when the safety comments are simply wrong
Really makes me wonder about the merits of requiring them on every unsafe if people just write wrong safety comments
@tender nimbus if you or anyone else fixes a SIGILL please shoot me a message or link me the PR or something, I want to track what bugs these are finding
ruby jacinth
Really makes me wonder about the merits of requiring them on every `unsafe` if p...
At least if the unsafe code itself looks OK at a glance but the reasoning in the safety comment is totally off, that'll make me look extra hard
haughty mica
🤷 Check out the above code from polars_io
It's a very common mistake but it's just a wrong safety comment
haughty mica
Yeah cool but if I view that web page I need you to know that I cannot figure out what is being run
If you could name the test that was still running maybe I could help, but this level of slowdown is typical
Const eval is ~1000x slowdown, and SB is ~infinite in the general case
tender nimbus
<@414755070161453076> if you or anyone else fixes a SIGILL please shoot me a mes...
You have found the SIGILL in the two swc crates yourself but here's the fix: <1https://github.com/swc-project/swc/pull/4943>
wtf, invalid pointer derefs inside vec in alacritty
tender nimbus
running just the data race test of arc-swap in miri makes it uaf instead
fun
TSan finds the data race as well
yikes
they do run tsan in ci though 
ah there is an issue
so they say that this is a false positive
tender nimbus
this goes way above my head
https://github.com/vorner/arc-swap/issues/76 i opened an issue i guess
haughty mica
I thought arc-swap fixed the SB issue but just didn't release it
tender nimbus
no
haughty mica
Oh I swear I did a PR, darn
knotty oar
i'm sure saethwin dreams about SB issues
haughty mica
I wish my dreams were that calm
tender nimbus
I wish my dreams were that calm
who doesn't
thank you discord
please stop using relative message counts for the replies
west violet
Yeah cool but if I view that web page I need you to know that I cannot figure ou...
Yah it’s kinda annoying that gh doesn’t show in-progress logs
But it’s the exchange test that it’s stalled on
west violet
Oh my bad I totally didn’t realize you had to log in to view logs
haughty mica
Because normally right these tests are pretty quick so it doesn't matter but doctests have that nice warning when they run for a while
west violet
I’ll send the log archive, gh doesn’t leak secrets in logs right?
Yah, nothing else has any trouble
haughty mica
Just paste me the name of the test that was stuck
west violet
Even all four of the sanitizers are super quick
haughty mica
Yeah they're only like 2x slowdown
tender nimbus
sanitizers slow a program down
unlike miri, which basically brings it to a halt
west violet
operator::communication::exchange::tests::test_exchange
Yah I was just confirming that this is very much a miri issue, nothing else cares
haughty mica
Yeah totally, that's why I think it would be best to do these sorts of hacks in cargo-miri
Oh this is SB thrashing
Just a perf top --pid $(pgrep miri | tail -n1)
tender nimbus
haughty mica
Those top 3 functions are linear in the runtime of the borrow stack
Total memory usage holding steady though that's nice
perf says 97% of runtime in SB, but because SB also trashes your cache it's closer to 100%
tender nimbus
is it better on your branch?
haughty mica
It's better
I think I never implemented a fix for the linear behavior of find_first_write_incompatible
This still destroys the cache, though I could fix that, though I think Ralf would be unhappy about it
Curious that this doesn't stress iter_mut
tender nimbus
This still destroys the cache, though I could fix that, though I think Ralf woul...
why, what would you do?
haughty mica
Well first of all this needs SoA
west violet
lmao
haughty mica
This loop is just a scan of Permission which is a 4-variant enum, but it's actually scanning a Vec<Item> where Item is 24 bytes
tender nimbus
tender nimbus
why would this make ralf unhappy? because it would make the code harder to read?
haughty mica
It makes the code harder to hack on
west violet
Dammit, why can't I remember the word?
It's for when there's a bad thing and you've found a situation that exacerbates it to horrible levels
haughty mica
The changes Ralf wants to make to SB are very much in the guts of how all this lookups into the borrow stacks work
west violet
It makes the code harder to hack on
If you make the api nice enough it could be a moot point?
west violet
It's for when there's a bad thing and you've found a situation that exacerbates ...
Specifically in software, it's not an edge case but close?
Dammit
tender nimbus
there was a soa derive crate that made this kind of not completely horrible i think?
west violet
Whatever, of course we made another piece of code that thrashes rust stuff lmao
haughty mica
SB is a big tangled mess of state so yes you could do that but it's not easy and I'm trying to do it
tender nimbus
why
haughty mica
Sounds like a good benchmark
west violet
I thought so too but no one took me up on it
tender nimbus
was the bug fixed
haughty mica
wg-compiler-perf didn't want it?
west violet
was the bug fixed
Donno
tender nimbus
or what caused this abomination
west violet
Lemme find the issue
Maybe it was this? https://github.com/rust-lang/rust/issues/78925
west violet
Totally, it's still a really big issue though
Other teams were disliking us because of our compile times
haughty mica
It's an issue with the architecture of your code
There's really not much the compiler could do to help you here
tender nimbus
the first timely test already times out in miri 
haughty mica
timely uses abomonation
west violet
Timely also has its own ub apart from abomonation
The consolidation code is ub off the top of my head
haughty mica
"potential" unaligned memory access
tender nimbus
no response for over 6 months
haughty mica
That's typical
west violet
Frank doesn't care
haughty mica
materialize isn't getting owned so...
west violet
Hum?
haughty mica
It's not a security issue
west violet
Ahh gotcha
I mean, abomonation is a massive issue if he ever wants to actually utilize timely's distribution mechanics
It will shit the bed, I've tried before
haughty mica
I honestly do not understand the use case for all this
west violet
For timely?
haughty mica
I've worked in code that needs to pump data to a serialization format and looking at the system holistically I would never use abomonation because that doesn't help with my hot path
west violet
Oh yah, same
haughty mica
the first `timely` test already times out in miri <:ferrisForgor:920768835135627...
times out?
btw I'm running it and this is in the interpreter not SB lol
west violet
It's vaguely hot since it's used within exchange operators, but they're miniscule in comparison to building/maintaining indexes and anything using indexes like joins or aggregation
haughty mica
For me (and maybe this is just because of our cost structure) arranging the data to be compressible and compressing it always dominates feeding it to a serializer
west violet
Also shilling dbsp, we're faster and use less memory than timely, and we actually have theory & math behind our stuff that's understandable
west violet
For me (and maybe this is just because of our cost structure) arranging the data...
Yep, most definitely
west violet
Also shilling dbsp, we're faster and use less memory than timely, and we actuall...
We're also working on disk-backed persistence and out-of-core indexes, meaning we can work over more data than main memory can hold which timely can't do since it's 100% in-memory
tender nimbus
Also shilling dbsp, we're faster and use less memory than timely, and we actuall...
but how can you be faster than timely, you don't even do horrible ub??? 🚀
west violet
but how can you be faster than timely, you don't even do horrible ub??? 🚀
Could my miri thing be because of thread::yield_now()?
Maybe that's screwing with miri
haughty mica
What is your miri thing
west violet
It taking multiple hours?
haughty mica
No, that's just Miri being slow and SB being ~infinitely slow
west violet
Ah gotcha
haughty mica
You need to shrink the working set of the test or cfg it out
tender nimbus
usually, there is literally nothing behind miri being slow
just.. miri being slow
west violet
I'm decreasing the number of rounds it does
tender nimbus
reducing the work miri will have to do will make the test faster
haughty mica
The dbsp SRW blocks are thousands of tags long 
west violet
You're welcome for your bench case
haughty mica
Pog
golden summit
tender nimbus
🎉
neon tiger
Minimal work done, maximal allowed concurrency
https://github.com/kolloch/crate2nix may get you this
caveat: it's Nix, so you'll have to dick around with it forever to get it to work
west violet
lol
neon tiger
but it will get you obviously-sound per-crate caching
obviously because the cache is keyed by all inputs, by construction
west violet
That could be good, yah
And I guess nix would help with the volunteer problem, people could just spin it up
haughty mica
I'm now obsessing about a good way to share build artifacts, thanks yall
tender nimbus
because it's cool
tough leaf
flurry has UB and the only reason it wasn't detected is because they cfg_attr(miri, ignore) some tests
:)
i'll look into this
it's at least a fresh form of UB (deallocation with wrong layout)
wait
it's a seize issue?
hmmm
yeah, sieze's tests fail miri
which is weird since they have miri in CI
tough leaf
well
maybe
i'm still looking into this
but yeah weird
oh
wtf
this is allocating a vec and then deallocating it with Box::from_raw ??
no
tough leaf
is what not allowed
wait
why is this making a vec
fn allocate_bucket<T>(size: usize) -> *mut Entry<T> {
Box::into_raw(
(0..size)
.map(|_| Entry::<T> {
present: AtomicBool::new(false),
value: UnsafeCell::new(MaybeUninit::uninit()),
})
.collect(),
) as *mut _
}
can you collect into a box?
tender nimbus
wtf is this
tough leaf
yes you can
tough leaf
impl<I> FromIterator<I> for Box<[I]> {
fn from_iter<T: IntoIterator<Item = I>>(iter: T) -> Self {
iter.into_iter().collect::<Vec<_>>().into_boxed_slice()
}
}
lmao
tough leaf
okay so
is Entry dynamically sized
how does it know the length
it is not
oh
is this
okay
it's allocating a *mut [Entry<T>] (or similar)
where the size is not 1
and then it's deallocating it as a *mut Entry<T>
pastel lily
ah
tender nimbus
tf
tough leaf
and that doesn't blow up in practice
because malloc/free does not care
about layout
tender nimbus
that's pretty cursed
tough leaf
but miri does
okay so
we do know how big the bucket we just made is
where is allocate_bucket used
i might just change that to return *mut [Entry<T>]
and then cast that to a *mut Entry<T> as needed
hmmm
nah i'll just deallocate using thread.bucket_size
how do you deallocate a boxed slice from a pointer
tough leaf
don't i need ptr metadata for that
tender nimbus
ptr::slice_from_raw_parts
tender nimbus
currently looking at some cursed alacritty shit
they read pointer bytes as normal bytes
not good
tough leaf
i might need to pass -Zmiri-allow-ptr-int-transmute for this to pass
since we might be putting pointer bytes in atomics
(integer atomics)
yep
tender nimbus
a lot better than the issue before
tough leaf
or not, actually?
tender nimbus
oh nice
tender nimbus
😵💫
tough leaf
strange
well
do the rest of the tests pass allowing ptr-int transmute
they do not
data race
deallocate / read
race
tender nimbus
ah, this uses a custom qword memcpy
but using usize instead of MU<usize>
tough leaf
MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-allow-ptr-int-transmute -Zmiri-disable-weak-memory-emulation -Zmiri-ignore-leaks"
tests pass with this
haughty mica
Could be the weak memory emulation bug again
tough leaf
is that a stdlib bug or a miri bug
tender nimbus
if we implement memcpy using MaybeUninit, is it mumcpy?
knotty oar
it's MaybeCopy
tender nimbus
tough leaf
hi
miri's complaining about the Box::from_raw
i can get (current) miri to pass with no errors but only if i enable like 3 ignores
manic tangle
that part of the code is pretty much just vendoring the thread-local crate
tough leaf
ah so thread-local is broken too?