Rename network_filter to duplicate_filter, rework its interface and keep cryptopp out of its header (#5264) * Rename network_filter to duplicate_filter Nothing in the class is specific to the network: it reports whether a digest was seen recently, forgets entries as they age or lose their slot, and erases them on request. Its config keys, tests and comments already call it the duplicate filter. The class, its files, its mutex id and the deserialize_message parameter take that name; the network keeps its member as filter. * Rename duplicate_filter::digest_t to digest The filter's digest type is now plain digest. The entry spells its digest member's type qualified, since GCC rejects a member declared with an unqualified type of the same name. * Use uint64_t for duplicate_filter epochs instead of epoch_t The alias added a name to look up without adding meaning, since an epoch is just a count the owner advances. The age cutoff, the current epoch, each entry's epoch and the call that advances them now use uint64_t directly. * Rename duplicate_filter check, clear and update to contains, erase and advance check () becomes contains (), the clear () overloads that remove one digest become erase (), and update () becomes advance (). Removing an entry no longer shares its name with clear (), which empties the whole table, and the epoch step says what it moves. Callers change their calls only. * Return true from duplicate_filter::insert when the digest is new apply () returned whether the digest was already in the filter, so a call that went through read as a failure: ASSERT_TRUE (filter.apply (bytes)) asserted a duplicate. It becomes insert () and returns true when the digest was new and went in, as message_processor::enqueue, block_processor::add and std::set::insert do; false marks a duplicate. The new name makes every caller change along with the meaning, and each one keeps checking what it checked before. * Take byte spans in duplicate_filter and drop the digest out-parameter The byte overloads take std::span<uint8_t const> instead of a pointer and a count, and insert () no longer hands back the digest through a pointer: deserialization hashes the payload with hash (), inserts the digest and keeps it for the message. The serializing template that was only instantiated for std::shared_ptr<nano::block> becomes hash () and erase () overloads taking a block, and the unused erase () over a vector of digests goes. The header is regrouped and its comments state each member's contract, including why entries age in epochs rather than clock time: it spares every insert a clock read. The tests are renamed and described, and new ones cover clear () and a block erasing the entry its publish went in under. * Keep cryptopp out of the duplicate_filter header The SipHash key is now a std::array filled from the random pool and wiped in the destructor, as CryptoPP::SecByteBlock did, and SipHash is only named in duplicate_filter.cpp. The header reaches most of the node through the publish and confirm_ack messages and the network, so those files no longer compile the cryptopp headers. test_common/system.cpp spelled CryptoPP::word32 through that include and now uses unsigned, the type random_pool::generate_word32 takes. * Benchmark duplicate_filter inserts with and without a clock read Entries age in epochs rather than clock time so that inserts never read the clock. The benchmarks measure what that saves: they insert payloads the size of a single-hash vote and of a state block publish to a filter of the node's default size, with and without a steady clock read before each insert, and the table side alone with precomputed digests, from one thread and from eight. * Check end to end that a filtered message passes again after the cutoff network.expire_duplicate_filter watched the entry leave the table but never sent the message again. It now resends the vote within the cutoff, where it must still be caught, checks that the entry ages out no sooner than the cutoff after the first copy was sent, and resends it once more, where it must be dispatched like a new message. The node's own cleanup loop advances the filter throughout. * Check that elections erase rejected and evicted forks from the filter active_elections.fork_filter_cleanup ended on a polled insert () that passes whether or not the fork was erased: when the fork is absent, the first poll inserts it and the second reports a duplicate. The test now runs on one node, enters the fork into the duplicate filter as a peer's publish of it would, lets the full election reject it and waits for the entry to go. Nothing checked the filter when an election evicts a fork to make room. The eviction fixture now takes how each fork reaches the node, with an overload that submits them as live blocks for the tests that do not care, and election.evicted_fork_leaves_duplicate_filter sends its forks through the filter and checks that the evicted fork is erased while the forks the election holds stay filtered.
__________
Type: commit
Hash: 634229949f905c41c67746843e10f64d5e9eb6c4
Created: today
__________
gr0v1ty/nano-node:634229949f905c41c67746843e10f64d5e9eb6c4
| Test Case | Status | Duration | Median Deviation | Log | Graph |
|---|---|---|---|---|---|
| 5n4pr_conf_10k_bintree | PASS | 126 s | 14.0 s | GitHub job logs | Show |
| 5n4pr_conf_10k_change | PASS | 176 s | 41.0 s | GitHub job logs | Show |
| 5n4pr_conf_change_dependant | PASS | 389 s | 272.0 s | GitHub job logs | - |
| 5n4pr_conf_change_independant | PASS | 112 s | 0.0 s | GitHub job logs | - |
| 5n4pr_conf_send_dependant | PASS | 124 s | 9.0 s | GitHub job logs | - |
| 5n4pr_conf_send_independant | PASS | 172 s | 59.0 s | GitHub job logs | - |
| 5n4pr_rocks_10k_bintree | PASS | 517 s | 405.0 s | GitHub job logs | - |
| 5n4pr_rocks_10k_change | PASS | 153 s | -5.0 s | GitHub job logs | - |