2024-07-12 –, Else (1.3)
The talk will be about Rembus: a new package for implementing distributed applications using a pub/sub communication style and/or a RPC communication style.
Rembus integrates into a single middleware the Publish/Subscribe and the Remote Procedure Call pattern: this is a unique feature amongst similar existing solutions (for example kafka, MQTT, grpc).
Rembus targets microservices oriented enterprise systems.
Main goals are:
- Make it easy implementing distributed applications avoiding the boilerplate of connections management.
- Overall performance.
- Broker and broker-less architecture.
- Exchange primitive types but also binary data and Dataframes objects between components.
Rembus leverages Julia's metaprogramming capabilities to propose a novel API approach for transforming a function into a "distributed" function, for example:
Pub/Sub: 1 publisher and N subscribers
The application "publisher" produces a message:
@component "publisher"
@publish mytopic(arg1, arg2, arg3)
NOTE: arguments
arg1, arg3, arg3are encoded as the data content of the published message with topicmytopic.
The application "consumer1" subscribes to the topic mytopic:
@component "consumer1"
# invoked for each received message on topic mytopic
function mytopic(arg1, arg2, arg3)
# body
end
@subscribe mytopic # declare interest to topic mytopic
The same hold true for "consumer2" .. "consumerN" applications each of them with specific body implementation of mytopic method.
Talk outline
The talk session will be organized as follows:
- Introduction: motivations and Rembus main features.
- Fault-tolerance features: brief explanation of a non trivial use of Visor.jl
- How to implements pub-sub and RPC systems with Rembus: code examples.
- Demo: exchanging dataframes between Julia, Python and Rust.
- Q&A
Senior Software Architect at TIM