Availability

• NATS will try to protect itself at all costs to be available for all users
• NATS client libraries, internally try to have an always established connection to one of the available NATS servers
• In case a server fails, NATS will reconnect to another available server in the pool
• NATS supports high-availablity via a clustering mode that is set up as a full-mesh of the servers

Subject Names

• The dot character (.) has special meaning as part of the subject.
• When using it as part of the subject, we can create namespaces (hierarchy) that can be later matched via wildcards.

• *: matches a single token
• >: matches one or more tokens in the subject, and it can only appear at the end

Worker 1 👷

INFO {"server_id":"NARXYLU3XEZSFYCFRV5WJ63GQS5WDV3WC4WAKCYDYY2IBGIETWJADZVS","server_name":"NARXYLU3XEZSFYCFRV5WJ63GQS5WDV3WC4WAKCYDYY2IBGIETWJADZVS","version":"2.1.8","proto":1,"git_commit":"c0b574f","go":"go1.14.8","host":"0.0.0.0","port":4222,"max_payload":1048576,"client_id":4,"client_ip":"172.22.0.1"}
SUB requests workers 5
+OK

MSG requests 5 6
second
    

Worker 2 👷

INFO {"server_id":"NARXYLU3XEZSFYCFRV5WJ63GQS5WDV3WC4WAKCYDYY2IBGIETWJADZVS","server_name":"NARXYLU3XEZSFYCFRV5WJ63GQS5WDV3WC4WAKCYDYY2IBGIETWJADZVS","version":"2.1.8","proto":1,"git_commit":"c0b574f","go":"go1.14.8","host":"0.0.0.0","port":4222,"max_payload":1048576,"client_id":5,"client_ip":"172.22.0.1"}
SUB requests workers 55
+OK

MSG requests 55 5
first
    

Publisher ✍️

INFO {"server_id":"NARXYLU3XEZSFYCFRV5WJ63GQS5WDV3WC4WAKCYDYY2IBGIETWJADZVS","server_name":"NARXYLU3XEZSFYCFRV5WJ63GQS5WDV3WC4WAKCYDYY2IBGIETWJADZVS","version":"2.1.8","proto":1,"git_commit":"c0b574f","go":"go1.14.8","host":"0.0.0.0","port":4222,"max_payload":1048576,"client_id":6,"client_ip":"172.22.0.1"}

PUB requests 5
first
+OK

PUB requests 6
second
+OK
    

Streaming

temporal decoupling between the publishers and subscribers

Subscribers only receive the messages that are published when they are actively connected to the messaging system (i.e. they do not receive messages that are published while they are not subscribing or not running or disconnected).

However, nowadays a new way to provide this temporal de-coupling has been devised and has become 'mainstream': streaming.

Streams capture and store messages published on one (or more) subject and allow client applications to create subscribers (i.e. JetStream consumers) at any time to replay (or consume) all or some of the messages stored in the stream

Replay policies

JetStream consumers support multiple replay policies.

• all of the messages currently stored in the stream, meaning a complete replay and you can select the replay policy(i.e. the speed of the replay) to be either:

instant

the messages are delivered to the consumer as fast as it can take them.

original

the messages are delivered to the consumer at the rate they were published into the stream, which can be very useful for example for staging production traffic.

• the last message stored in the stream, or the last message for each subject (as streams can capture more than on subject).
• starting from a specific sequence number.
• starting from a specific start time.

Retention policies and limits

limit

You can impose the following limits on a stream

• Maximum message age
• Maximum total stream size (in bytes)
• Maximum number of messages in the stream
• Maximum individual message size
• You can also set limits on the number of consumers that can be defined for the stream at any given point in time.

You must also select a discard policy which specifies what should happen once the stream has reached one of its limits and a new message is published

• discard old means that the stream will automatically delete the oldest message in the stream to make room for the new messages.
• discard new means that new message is discarded (and the JetStream publish call retuns an error indicating that a limit was reached)

Retention policy

You can choose what kind of retention you want for each stream

• limits (the default)
• interest (messages are kept in the stream for as long as there are consumers that haven't delivered the message yet).
• work queue (the stream is used as a shared queue and messages are removed from it as they consumed)

Regardless of the retention policy selected, the limits (and the discard policy) always apply.

Persistent distributed storage

You can choose the durability as well as the resilience of the message store according to your needs.

• Memory storage
• File storage
• Replication (1 (none), 2, 3) between nats servers for Fault Tolerance

JetStream uses a NATS optimized RAFT distributed quorom algorithm to distribute the persistence service between nats servers in a cluster while maintaining immediate consistency even in the face of Byzantine failures.

In JetStream the configuration for storing messages is defined separately from how they are consumed.

Stream replication factor

A stream's replication factor (R, often referred to as the number Replicas) determines how many places it is stored allowing you to tune to balance risk with resource usage and performance.

A stream that is easily rebuilt or temporary might be memory based with a R=1 and a stream that can tolerate some downtime might be file based with R=1.

Exactly once semantics

Because publications to streams using the JetStream publish calls are acknowledged by the server, the base quality of service offered by streams is at least once.

Meaning that while reliable and normally duplicate free there are some specific failure scenarios:

• That could result in a publishing application believing (wrongly) that a message was not published successfully and therefore publishing it again.
• That could result in a client application's consumption acknowledgement getting lost and therefore the message being re-sent to the consumer by the server.

Therefore, JetStream also offers an exactly once.

• For the publishing side it relies on the publishing application attaching a unique message or publication id in a message header and on the server keeping track of those ids for a configurable rolling period of time in order to detect the publisher publishing the smae message twice.
• For the subscribers a double acknowledgement mechanisim is used to avoid a message being erroneously re-sent to a subscriber by the server after some kinds of failures

References 📚

• Practical NATS: From Beginner to Pro, Book by Waldemar Quevedo
• NATS Documentation