4 The high level goal is to transmit messages between brokers without
5 requiring clustering. This is useful for the following reasons:
7 1) Federated brokers may be in different administrative
8 domains. Clustered brokers form a single administrative domain.
10 a) Federated brokers may have different users and virtual hosts.
11 Federated brokers only need to partially trust each other.
13 b) Federated brokers may run different versions of RabbitMQ and
16 2) Federated brokers only speak AMQP to each other, and the federation
17 mechanism is designed to deal with intermittent connectivity.
18 Federation is therefore much more WAN-friendly.
20 3) Brokers can contain federated and local-only components - you don't
21 need to federate everything if you don't want to.
23 4) Ultimately, greater scalability should be possible, since more complex
24 routing topologies avoid the need for n^2 connections between n
27 For the time being, federation is primarily useful in pub/sub scenarios.
33 You can't federate headers exchanges. You can't make a broker federate
34 with a new upstream without restarting it. There's no status
35 reporting. These will likely get fixed.
41 The plugin provides a federated exchange type. A federated exchange
42 has type 'x-federation'. However, this type does not provide any
43 routing logic. The routing logic is implemented by a backing type,
44 provided to the exchange as an argument.
46 Messages can be published to a federated exchange like any
47 other. However, a federated exchange also receives messages from
48 one or more "upstream" exchanges, located on remote brokers.
50 (Well, you don't need to have upstream exchanges, and they don't need
51 to be remote. But then you don't get anything very useful.)
53 An upstream exchange can be a regular exchange or a federation
54 exchange. It is expected that upstream and downstream exchanges have
55 the same type (or backing type). Mixing types will lead to strange
58 Inter-broker communication is implemented using AMQP (optionally
59 secured with SSL). Bindings are grouped together and bind / unbind
60 commands are sent to the upstream exchange. Therefore the federation
61 plugin only receives messages over the wire for which the downstream
62 exchange has a subscription. To promote scalability the bindings are
63 sent upstream asynchronously - so the effect of adding or removing a
64 binding is only guaranteed to be seen eventually.
66 A typical use would be to have the same "logical" exchange distributed
67 over many brokers. This would be achieved by having the exchange
68 declared as a federated exchange in each broker, with upstreams
69 corresponding to all the other brokers.
71 Another would be massive fanout - a single "root" exchange in one
72 broker can be treated as an upstream by many other brokers. In turn
73 each of these can be the upstream for many more downstreams, and so
76 Other topologies are of course possible.
78 Federated exchanges can be set up statically via broker configuration,
79 or declared dynamically over AMQP.
85 A verbose static configuration might look like:
88 [ {exchanges, [[{exchange, "my-exchange"},
94 {upstream_set, "my-upstreams"}]
96 {upstream_sets, [{"my-upstreams", [[{connection, "upstream-server"},
97 {exchange, "my-upstream-x"},
99 [{connection, "another-server"}]
102 {connections, [{"upstream-server", [{host, "upstream-server"},
106 {username, "myusername"},
107 {password, "secret"},
108 {mechanism, default},
109 {prefetch_count, 1000},
110 {reconnect_delay, 5},
112 {queue_expires, 30000},
113 {message_ttl, 10000},
115 [{cacertfile, "/path/to/cacert.pem"},
116 {certfile, "/path/to/cert.pem"},
117 {keyfile, "/path/to/key.pem"},
118 {verify, verify_peer},
119 {fail_if_no_peer_cert, true}
122 {"another-server", [...elided...]}
124 {local_username, "myusername"},
125 {local_nodename, "my-server"}
129 The list of exchanges looks like a set of exchange.declares for the
130 most part, but with each declaration including the name of an
131 "upstream_set", representing a list of remote exchanges whose messages
132 should be federated to the local exchange. Note that the type
133 parameter must match the type of the upstream exchanges for routing to
134 work at all sensibly.
136 Each element in the upstream_set contains a mapping from a name to a
137 list of upstreams. Each element in this list can contain the following
141 - name of a connection from the connection list. Mandatory.
144 - name for exchange to connect to. Default is to use the same name
145 as the downstream exchange. (Therefore one upstream_set can be
146 refered to by many local exchanges as long as exchange names are
147 the same across all the nodes in your federation.)
150 - the maximum number of times a message received over this link may
151 have been forwarded (including traversing this link). The default
152 for max_hops is 1. This prevents messages from looping forever
153 when there are circular topologies, and can reduce or prevent
156 Note that the static configuration will declare the downstream
157 exchanges (on the local broker). It does not ensure the upstream
160 The connections list provides information on how to connect to brokers
161 mentioned in the upstream sets. It can contain the following properties:
164 - hostname to connect to
167 - "amqp" or "amqps". Default is "amqp".
170 - port to connect to. Default is 5672, or 5671 when using SSL.
173 - virtual host to connect to. Default is to use the virtual host for
174 the downstream exchange.
177 - user to connect as. Default is "guest". The user will need the
178 ability to create exchanges and queues with names beginning with
179 "federation:". It will also need to be able to bind to the upstream
183 - password to use. Default is "guest".
186 - SASL mechanism to use. One of:
187 'default' - to use PLAIN or AMQPLAIN by negotiation (this is the default)
188 'EXTERNAL' - to use SASL EXTERNAL authentication - e.g.
189 rabbitmq-auth-mechanism-ssl
192 - limit on the maximum number of unacknowledged messages in flight
193 per link. Default is 'none'.
196 - time in seconds to wait to reconnect to the broker after being
197 disconnected. Default is 1.
200 - AMQP heartbeat interval (in seconds) on the connection, or none. Default
204 - Messages are buffered in the upstream broker in a queue before being
205 sent downstream. This setting controls how long the upstream queue
206 lasts before being deleted if the downstream is disconnected (i.e.
207 it controls the "x-expires" argument for the upstream queue). The
208 value is in milliseconds. Default is 'none', meaning the queue should
212 - Controls how long to keep upstream messages buffered, in milliseconds
213 (i.e. the "x-message-ttl" argument for the upstream queue). Default is
214 'none', meaning messages should never expire.
217 - Specifies how to make client SSL connections. See the Erlang
218 client documentation for more details.
220 The local_username parameter specifies how to connect to the local
221 broker. The default is "guest". This user will need the ability to
222 publish messages to the downstream exchange.
224 The local_nodename parameter specifies the name this node should use
225 to identify itself in the federation. If not specified it will attempt
226 to build a "long form" version of the usual Erlang node name (but
227 using the machine's FQDN). You should only have to specify this if
228 your DNS will not give each machine in the federation unique names.
230 Declaring Federation Exchanges Over AMQP
231 ========================================
233 This is a less common case, but in case you want to do this:
235 * Declare the downstream exchange with type "x-federation".
237 * Give it arguments "type" and "upstream-set", both of type "long
238 string". "type" should refer to its backing type; the type of the
239 upstream exchanges. "upstream-set" should be the name of an
240 upstream_set from the static configuration (note that it's a hyphen
241 over AMQP but an underscore in the configuration).
243 Example using the Java API:
245 Map<String, Object> args = new HashMap<String, Object>();
246 args.put("type", "topic");
247 args.put("upstream-set", "my-upstream-set");
250 ch.exchangeDeclare("my-federated-exchange", "x-federation", true, false, args);
255 When using federation with clustering, all cluster nodes must have the
256 federation plugin installed. Any node may establish connections to
257 upstream exchanges. If a node fails, any upstream links will fail over
260 Possible Future Enhancements
261 ============================
263 * Simple status reporting, similar to the shovel
265 * Modify upstream sets dynamically
267 * Mode to federate all exchanges
269 * Status information and control in the management plugin
271 * Smarter routing logic
273 * Support for RPC-like patterns