What happened is that we created a self-referential struct, initialized it so that it took a pointer to itself, and then moved it. Let’s take a closer look:

1. First, we received a future object as an argument to block_on. This is not a problem since the future isn’t self-referential yet, so we can move it around wherever we want to without issues (this is also why moving futures before they’re polled is perfectly fine using proper async/await).
2. Then, we polled the future once. The optimization we did made one essential change. The future was located on the stack (inside the stack frame of our block_on function) when we polled it the first time.
3. When we polled the future the first time, we initialized the variables to their initial state. Our writer variable took a pointer to our buffer variable (stored as a part of our coroutine) and made it self-referential at this point.
4. The first time we polled the future, it returned NotReady
5. Since it returned NotReady, we spawned the future, which moves it into the tasks collection with the HashMap<usize, Box<dyn Future<Output = String>>> type in our Executor. The future is now placed in Box, which moves it to the heap.
6. The next time we poll the future, we restore the stack by dereferencing the pointer we hold for our writer variable. However, there’s a big problem: the pointer is now pointing to the old location on the stack where the future was located at the first poll.
7. That can’t end well, and it doesn’t in our case.

You’ve now seen firsthand the problem with self-referential structs, how this applies to futures, and why we need something that prevents this from happening.

A self-referential struct is a struct that takes a reference to self and stores it in a field. Now, the term reference here is a little bit unprecise since there is no way to take a reference to self in Rust and store that reference in self. To do this in safe Rust, you have to cast the reference to a pointer (remember that references are just pointers with a special meaning in the programming language).

Note

When we create visualizations in this chapter, we’ll disregard padding, even though we know structs will likely have some padding between fields, as we discussed in Chapter 4.

When this value is moved to another location in memory, the pointer is not updated and points to the “old” location.

If we take a look at a move from one location on the stack to another one, it looks something like this:

Figure 9.1 – Moving a self-referential struct

In the preceding figure, we can see the memory addresses to the left with a representation of the stack next to it. Since the pointer was not updated when the value was moved, it now points to the old location, which can cause serious problems.

Note

It can be very hard to detect these issues, and creating simple examples where a move like this causes serious issues is surprisingly difficult. The reason for this is that even though we move everything, the old values are not zeroed or overwritten immediately. Often, they’re still there, so dereferencing the preceding pointer would probably produce the correct value. The problem only arises when you change the value of x in the new location, and expect y to point to it. Dereferencing y still produces a valid value in this case, but it’s the wrong value.

Optimized builds often optimize away needless moves, which can make bugs even harder to detect since most of the program will seem to work just fine, even though it contains a serious bug.