Messages

ERDOS applications send data on streams via messages. Messages wrap data and provide timestamp information used to resolve control loops and track data flow through the system.

class erdos.Message(timestamp, data)

Class used to wrap ERDOS message data.

timestamp

The timestamp of the message.

Type

Timestamp

data

The data of the message.

Timestamps

Timestamps consist of an array of coordinates. Timestamp semantics are user-defined for now; however, we may eventually formalize their use in the future in order to provide more advanced features in order to scale up stateful operators. Generally, the 0th coordinate is used to track message’s sequence number and subsequent coordinates track the message’s progress in cyclic data flows.

class erdos.Timestamp(timestamp=None, coordinates=None, is_top=False)

An ERDOS timestamp. The timestamp can consist of one or more coordinates.

timestamp

For the copy constructor.

Type

Timestamp

coordinates

An array of coordinates.

Type

list of int

is_top

Whether this is the highest possible timestamp.

Type

bool

Watermarks

Watermarks in ERDOS signal completion of computation. More concretely, sending a watermark with timestamp t on a stream asserts that all future messages sent on that stream will have timestamps t' > t. ERDOS also introduces a top watermark, which is a watermark with the maximum possible timestamp. Sending a top watermark closes the stream as there is no t' > t_top, so no more messages can be sent.

class erdos.WatermarkMessage(timestamp)

Class used to wrap ERDOS watermark message.

timestamp

The timestamp for which this is a watermark.

Type

Timestamp