implement intptrcast model

[oli-obk]

Since right now miri's pointer comparison bails out on &x as *const T < &y as *const T but allows &x as *const T == &y as *const T, I suggest to also allow <. This will not impact ctfe-mode, since ctfe mode disallows any binary ops on pointers into different allocs, even ==.

It's deterministic between multiple miri runs, doesn't leak HashMap ordering and only depends on optimizations and codegen from rustc (but that's already true for ==).

Steps to do:

• Implement force_bits, force_int and basic address assignment.
• force_bits when doing primitive operations.
• force_bits when doing casts.
• force_ptr when creating a non-ZST reference. (Ideally through check_ptr_access? Redundant with validity checks but whatever.) We may need a force_ptr_place method or so; the freeze-sensitive visitor also needs that.
• Make force_bits infallible (in Miri, not CTFE) -- see Make force_ptr a total operation, and ptr-to-int-casts not require extra machine state #841.
• force_bits when doing partial loads of a pointer -- and really everywhere we need a ptr, once it is total. (For now let's not do this, also see const-eval: load of partially initialized scalar produces entirely uninitialized result rust#69488 for a similar problem around partially initialized data.)
• Exploit "real" pointer alignment for alignment checks, but warn when we do. (But also see align_offset guarantees rust#62420.)
• Ptr-to-int of a dangling pointer (possible in safe code).
• Fix the FIXMEs added in Add tests for Intptrcast when doing explicit casts #803.
• Audit all remaining uses of to_bits (this includes the to_<int type> methods...), to_ptr and to_bits_or_ptr, all uses of is_ptr and is_bits, and also all uses of force_ptr and force_mplace_ptr (also mplace_access_checked, check_mplace_access) outside of check_ptr_access. Maybe replace to_{bits,ptr} by assert_{bits,ptr}?

[RalfJung]

My plan was to go even further and essentially implement the intptrcast-model. That would immediately make us support all pointer arithmetic.

Essentially, I think at some point miri should (in non-CTFE mode!) guarantee that safe Rust code can not error out (if the unsafe libraries it uses don't contain bugs). So hashing raw pointers and printing them and stuff like that all have to work.

[oli-obk]

Sounds reasonable to me. Once a pointer has been observed and thus it's integral representation is available, there's no way in safe code to go from that pointer back to the allocation, so we don't even need to support that direction, right? All we need to ensure is hat the integral representation doesn't change when obtaining it multiple times from a real pointer

We'll also need to generate an integer address when offsetting pointers due to overflow concerns. Overflowing exposes the address essentially.

[RalfJung]

We'll also need to generate an integer address when offsetting pointers due to overflow concerns. Overflowing exposes the address essentially.

That's only true when you do the offsetting after casting to integer though, is it? The offset operations on pointers don't let safe code observe whether an overflow actually happened. Really, we could just generate an integer address immediately on allocation.

Doing lookup on a Pointer is still more efficient, so that may be a reason to convert back.

[oli-obk]

Well... You can do wrapping_offset (safe and stable method), and then do a <= comparison between the resulting pointer and the new one.

let x = 5;
for i in 1.. {
    let ptr = &x as *const i32;
    let ptr2 = ptr.wrapping_offset(i);
    if ptr2 <= ptr {
        println!("Address of `x` is {}", std::usize::max_value() - i);
    }
}

Really, we could just generate an integer address immediately on allocation.

We'd run out of integer addresses eventually (and quickly on 32bit emulation) if we don't reuse integer addresses and create a unique integer address for every allocation.

[RalfJung]

Heh, nice catch.

We'd run out of integer addresses eventually (and quickly on 32bit emulation) if we don't reuse integer addresses and create a unique integer address for every allocation.

"Quickly"? 2^32 is still a large number, I feel.^^
But anyway, I guess there's no problem with lazy allocation. That also gives us a nice single point of control for rejecting everything that could potentially leak the base address when in CTFE mode -- we just always make the allocation of an integer address fail in that mode.

[oli-obk]

The problem is, that once you leak the base address of an array, you also leak the address of the last element's last byte. So allocatingand deallocating 100MB 50 times would fill the address space unless we allow reusing it.

[RalfJung]

Actually, given what I wrote in https://internals.rust-lang.org/t/types-as-contracts/5562/81, I think it would make sense for miri to both support pointer comparisons on "abstract" pointers and only provide intptrcast optionally. Without intptrcast, there will always be safe code that miri cannot run (like printing the value of a pointer), but currently there still seems to be value in also having a mode that keeps pointers and integers more separate.

[Amanieu]

This code currently fails on miri:

fn main() {
    let x: Vec<u32> = Vec::new();
    &x == &[];
}

error[E0080]: constant evaluation error: attempted to do invalid arithmetic on pointers that would leak base addresses, e.g., comparing pointers into different allocations
    --> /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/slice/mod.rs:5024:12
     |
5024 |         if self.as_ptr() == other.as_ptr() {
     |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ attempted to do invalid arithmetic on pointers that would leak base addresses, e.g., comparing pointers into different allocations
     |
     = note: inside call to `<[A] as core::slice::SlicePartialEq<A>><u32>::equal` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/slice/mod.rs:4962:9
     = note: inside call to `core::slice::<impl std::cmp::PartialEq<[B]> for [A]><u32, u32>::eq` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/liballoc/vec.rs:2069:50
     = note: inside call to `<std::vec::Vec<A> as std::cmp::PartialEq<[B; _]>><u32, u32>::eq` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libcore/cmp.rs:1009:48
note: inside call to `std::cmp::impls::<impl std::cmp::PartialEq<&'b B> for &'a A><std::vec::Vec<u32>, [u32; 0]>::eq` at src/main.rs:3:5
    --> src/main.rs:3:5
     |
3    |     &x == &[];
     |     ^^^^^^^^^
     = note: inside call to `main` at /root/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/src/libstd/rt.rs:64:34