RCUObject.h File Reference

read-copy-update (RCU) style synchronization for Athena. More...

#include "GaudiKernel/ThreadLocalContext.h"
#include "GaudiKernel/EventContext.h"
#include "boost/dynamic_bitset.hpp"
#include <atomic>
#include <deque>
#include <vector>
#include <memory>
#include <mutex>
#include "AthenaKernel/RCUObject.icc"

Include dependency graph for RCUObject.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	Athena::RCURead< T >
	Helper to read data from a RCUObject. More...
class	Athena::RCUReadQuiesce< T >
	Helper to read data from a RCUObject. More...
class	Athena::RCUUpdate< T >
	Helper to update data in a RCUObject. More...
class	Athena::IRCUObject
	Base object class for RCU-style synchronization for Athena. More...
class	Athena::RCUObject< T >
	Wrapper object controlled by RCU synchonization. More...
class	Athena::RCURead< T >
	Helper to read data from a RCUObject. More...
class	Athena::RCUReadQuiesce< T >
	Helper to read data from a RCUObject. More...
class	Athena::RCUUpdate< T >
	Helper to update data in a RCUObject. More...

Namespaces
	Athena
	Some weak symbol referencing magic...

Detailed Description

read-copy-update (RCU) style synchronization for Athena.

Author

scott snyder snyde.nosp@m.r@bn.nosp@m.l.gov

Date

Aug, 2016 Read-copy-update (RCU) is a style of synchronization that is appropriate to a read-often, update-seldom case. More precisely:

• Reads are frequent and low overhead is important.
• Updates are infrequent, and extra overhead doesn't matter much.
• Exact time ordering between reads and updates is not important. That is, if a read happens at about the same time as an update, it doesn't matter if the read sees the data before or after the update (as long as the reader sees something that is consistent).

For a more detailed introduction see, for example:

https://lwn.net/Articles/262464 https://lwn.net/Articles/573424

There are many different styles of RCU implemenation; the one we implement here is close to the ‘quiescent-state-based RCU’ (QSBR) variant discussed in the second article above. Here, readers take out no locks whatsoever. It follows then that the updater cannot directly change the object that is being read. Instead, it makes an updated copy of the object and then publishes it by atomically changing a pointer to the object. The remaining question is then when can the old object be deleted. It is necessary to delay this until until no thread can be reading the old version of the object.

In this implementation, the updater does not wait for this to happen. Instead, after the update, each event slot (other than the updater) is marked as being in a ‘grace’ period (via an array of flags in RCUObject). A grace period ends for a given event slot when the object is declared ‘quiescent’; that is, it is at that time not referencing the object. Once the grace periods for all event slots have finished, then old versions of the object can be deleted.

It is possible to explicitly declare an object as quiescent, either by calling the quiescent() method or by using RCUReadQuiesce. Objects can also be registered with the Athena RCUSvc, which will declare objects quiescent at the end of an event.

The current implementation is quite simple, mostly suitable for a small number of objects that change infrequently. There was an issue whereby if objects are being updated frequently and one is relying on RCUSvc to mark them as quiescent, then it's possible that one will never be out of the grace period for all event slots simultaneously, preventing objects from being cleaned up. This is addressed by maintaining both a ‘current’ and ‘old’ set of grace periods. If an object is updated while there are still objects pending deletion, the pending objects are moved to an ‘old’ list, and we reset the grace periods only for the current list. We then don't move anything else to the old list until those objects have been deleted.

It still may be the case, however, that if there are a large number of infrequently-changing objects, then RCUSvc may waste time marking as quiescent objects that have not changed.

Usage:

The usual usage is to create an RCUObject instance via the RCUSvc. RCUObject is templated on the type of the controlled object.

ServiceHandle<Athena::RCUSvc> rcusvc ("Athean::RCUSvc", "test");
std::unique_ptr<Athena::RCUObject<Payload> > =
  rcusvc->newrcu<Payload>();

newrcu may also take additional arguments that will be passed to the constructor for Payload. This will register the object with the service. One can alternately create a RCUObject directly, passing it the number of event slots.

It is also possible to create a RCUObject that does not hold an object by passing IRCUObject::NoObject to the constructor. This is useful if the RCUObject is just being used to manage deletion of objects owned elsewhere.

To read the controlled object, create a RCURead object:

const Athena::RCUObject<Payload>& rcuobj = ...;
{
  Athena::RCURead<Payload> r (rcuobj);
  ret = r->someMethodOnPayload();
}

and to update use RCUUpdate:

Athena::RCUObject<Payload>& rcuobj = ...;
{
  Athena::RCUUpdate<Payload> u (rcuobj);
  auto newPayload = std::make_unique<Payload> (*u);
  newPayload->updateSomeStuff();
  u.update (std::move (newPayload));
}

To explicitly declare an object as quiescent, call the quiescent method. This takes an EventContext object, or it can be defaulted, in which case the current global context object will be used. You can also use RCUReadQuiesce instead of RCURead to automatically call quiescent when the read is complete (when the RCUReadQuiesce object is destroyed).

Definition in file RCUObject.h.