Previously, the chunk data cache was regenerated at the end of each tick
every time a block or biome was modified in a chunk. This causes
performance problems in situations where chunks are being modified every
tick, like the conway example or redstone machines.
This changes things so that the chunk data packet is regenerated only
when a client actually needs to load the chunk. This isn't perfect
because the regeneration cannot happen in parallel, but the benefits
outweigh the costs. (The chunk data packet _is_ generated in parallel
when the chunk is first created. There is also some parallelism when
clients are loading different chunks. The packet cache is guarded by a
`Mutex`.)
This has revealed that compression is surprisingly slow. This should be
investigated later.
Closes#84
# Changes
- Implemented new algorithm for tracking loaded entities and chunks.
- The client does not need to maintain a list of loaded chunks and
entities anymore, as this information can be inferred using carefully
maintained data from the previous tick.
- Chunks are used as a spatial partition for entities. Entity visibility
is now based on chunk visibility rather than the euclidean distance to
clients.
- The BVH is no longer strictly necessary, so it has been moved to the
new `valence_spatial_index` crate.
- The API has been generalized to support things other than entities.
The crate does not have a dependency on the main `valence` crate.
- Chunk, entity, and player list packets are now aggressively cached to
increase performance.
- Chunk packets now include some filler light data. This makes the
vanilla client lag a lot less.
- Entities and chunks must now be marked as deleted before they are
removed.
- Improved `ChunkPos` interface.
- Added function to get the duration per tick.
- Added `Index` and `IndexMut` impls to collection types.
As a result of the above changes, performance under heavy load has
increased significantly. With the rust-mc-bot test on my machine, I went
from a max of ~1000 players to ~4000 players.
# Noteworthy Changes
- Simplified the `client` module and update procedure. Sending packets
is not deferred unless necessary.
- Client events are no longer buffered in `Client` before reaching the
user.
- Expanded `ClientEvent` to account for most packets.
- Most types containing custom `state` now implement `Deref` and
`DerefMut`. This means you don't have to write `.state` all over the
place. `Server` was excluded from this because it does not play well
with the borrow checker.
- Fixed bugs related to entity visibility.
- Client now correctly holds the semaphore permit from the initial
connection.
- Other miscellaneous API changes throughout the project.
# Known Issues
- Inventory stuff is still incomplete. The inventory examples have been
temporarily disabled.
The `tracing` crate seems to be the go-to
logging/profiling/instrumentation solution nowadays. Perhaps in the
future we could use `tracing` for profiling instead of (or in addition
to) the `perf`-based `cargo flamegraph` command. This would sidestep the
issue of `rayon` polluting the output. I conducted an initial experiment
by adding some more spans but wasn't very happy with the result.
Log messages have also been improved. There is some additional context
and events are raised when clients are added/removed from the server.
With the introduction of `valence_protocol` and some future plans, I've decided it's unrealistic to try and hide all the protocol code. There are situations where direct access is necessary.
Closes#83
This PR aims to move all of Valence's networking code to the new
`valence_protocol` crate. Anything not specific to valence is going in
the new crate. It also redesigns the way packets are defined and makes a
huge number of small additions and improvements. It should be much
easier to see where code is supposed to go from now on.
`valence_protocol` is a new library which enables interactions with
Minecraft's protocol. It is completely decoupled from valence and can be
used to build new clients, servers, tools, etc.
There are two additions that will help with #5 especially:
- It is now easy to define new packets or modifications of existing
packets. Not all packets need to be bidirectional.
- The `CachedEncode` type has been created. This is used to safely cache
redundant calls to `Encode::encode`.
The `PlayPacketController` might spawn a tokio task, but it was not in a
tokio context when that happened. This is because tokio contexts are
thread-local, but rayon tasks run in their own thread pool.
- rework `consume_one` into `consume`
- update `consume_one_held_item` to `consume_held_item` to match
`consume`
- remove an unnecessary clone
Originally, I was planning on removing this function altogether. After
thinking about it for a bit though, I think it provides enough of a
convenience for end users to warrant keeping it.
This also implements the fix that was originally in #124
Closes#82Closes#43Closes#64
# Changes and Improvements
- Packet encoding/decoding happens within `Client` instead of being sent
over a channel first. This is better for performance and lays the
groundwork for #83.
- Reduce the amount of copying necessary by leveraging the `bytes` crate
and recent changes to `EncodePacket`. Performance is noticeably improved
with maximum players in the `rust-mc-bot` test going from 750 to 1050.
- Packet encoding/decoding code is decoupled from IO. This is easier to
understand and more suitable for a future protocol lib.
- Precise control over the number of bytes that are buffered for
sending/receiving. This is important for limiting maximum memory usage
correctly.
- "packet controllers" are introduced, which are convenient structures
for managing packet IO before and during the play state.
- `byte_channel` module is created to help implement the
`PlayPacketController`. This is essentially a channel of bytes
implemented with an `Arc<Mutex<BytesMut>>`.
- Error handling in the update procedure for clients was improved using
`anyhow::Result<()>` to exit as early as possible. The `client` module
is a bit cleaner as a result.
- The `LoginPlay` packet is always sent before all other play packets.
We no longer have to worry about the behavior of packets sent before
that packet. Most packet deferring performed currently can be
eliminated.
- The packet_inspector was rewritten in response to the above changes.
- Timeouts on IO operations behave better.
# Known Issues
- The packet_inspector now re-encodes packets rather than just decoding
them. This will cause problems when trying to use it with the vanilla
server because there are missing clientbound packets and other issues.
This will be fixed when the protocol module is moved to a separate
crate.
There are a number of places where usernames are passed around. Using this type ensures that the contained string is actually a valid username and not some other kind of string. For instance you can use it as a function argument to indicate that only valid usernames are accepted, or return it from a function to indicate that only valid usernames are produced.
This is analogous to the existing `Ident<S>` type.
Closes#114
Note that this is a band-aid solution to a more general problem. In the
future, there should be a function in the `block` module to handle all
these placement rules.
This allows packets to calculate their exact length up front.
This isn't currently tested or being used for anything, but that will come in later changes.
This is a very rough version of the inventory module and associated features. This will be a good base to start building something more robust.
- adds a player inventory to all clients
- makes clients keep track of what inventory they have open
- adds very naive state sync for inventories
related: #53
# Test plans
### Building
1. run `building` example
2. pick any block from creative
3. place it
4. pick a different block from creative
5. place 2
6. break the last one
7. see that both remaining blocks don't become dirt
### Chest inventories
1. run `chest` example
2. sneak to give yourself a stone block
3. open the chest (it's invisible on top of the stone block)
4. place stone in chest
5. see the stone moving around in the chest
## Known Issues
To be fixed in later PRs
- It's very easy to duplicate items if you take the stone out of the chest on the same tick as the server trying to move the item.
- The chest is invisible when you spawn in. It appears if you break it.
- `get_slot` and `set_slot` can panic.
Adds the performance_tests/ directory.
In the future we could use our own fake client software instead of
rust-mc-bot. This would make it easier to run the tests.
* `RespawnRequest` client event
* `CombatDeath` packet
* `kill` client method
* `death.rs` example
* clippy lint fix
* Clean up examples
* Fix formatting
* `win_game`, `set_respawn_screen` client methods
* respawn screen no longer deferred
This means that death example does not have respawn screen now.
Co-authored-by: Ryan <ryanj00a@gmail.com>
