name: inverse class: middle, bg-red, bg-dark layout: true --- <img src="figures/rwthlogo.png" height="60"> # gr-zmqblocks ### An approach to distributed GNU Radio applications __Johannes Schmitz__ <img src="figures/tilogo.png" height="30"> - Institute for Theoretical Information Technology October 2nd, 2013 [https://github.com/iohannez/gr-zmqblocks](https://github.com/iohannez/gr-zmqblocks) --- layout: false ## Motivation - replace CORBA in our OFDM System - we had this ControlPort like extension back in 2009 never released to the public :/ - build something simple, no modifications to GNU Radio core - enable distributed GNU Radio flowgraphs - enable remote GUIs _(display signals, send control messages_) - enable feedback loop for research on adaptive systems <center><img src="figures/signature_system.svg" width="300"></center> --- ##The enabling library - [http://zeromq.org/](http://zeromq.org/) - Think of ZeroMQ as some kind of "super sockets" - A lightweight library to implement distributed applications - no broker - Bindings for 40+ languages - Again: trying to keep it lightweight, e.g. no broker, no unnecessary stuff --- ## Some valuable resources I used: - [http://api.zeromq.org/](http://api.zeromq.org/) - [http://zguide.zeromq.org/](http://zguide.zeromq.org/) - [https://learning-0mq-with-pyzmq.readthedocs.org](https://learning-0mq-with-pyzmq.readthedocs.org/) - [http://blog.pythonisito.com/2012/08/distributed-systems-with-zeromq.html](http://blog.pythonisito.com/2012/08/distributed-systems-with-zeromq.html) - [http://blog.pythonisito.com/2012/08/zeromq-flow-control-and-other-options.html](http://blog.pythonisito.com/2012/08/zeromq-flow-control-and-other-options.html) --- ## Now let's have a look at the very basic ZeroMQ examples and fundamentals --- ## Server import zmq context = zmq.Context() socket = context.socket(zmq.REP) socket.bind("tcp://*:5555") # wait for request from client message = socket.recv() print "Received request: ", message # send reply back to client socket.send("World") --- ## Client import zmq context = zmq.Context() socket = context.socket(zmq.REQ) socket.connect ("tcp://localhost:5555") # send request socket.send ("Hello") # get the reply message = socket.recv() print "Received reply: ", message --- ## Demo --- ## ZeroMQ Socket Types / Messaging Patterns ## Pair <img src="figures/socket_pair.png" width="500"> - similar to regular sockets - bidirectional --- ## REQ REP <img src="figures/socket_reqrep.png" width="500"> - one request to many reply sockets possible - each request message has to be answered with a reply - otherwise blocking behaviour --- ## PUB SUB <img src="figures/socket_pubsub.png" width="700"> - publisher has no knowledge about subscribers - messages dropped if no subscriber or rate to high --- ## PUSH PULL <img src="figures/socket_pushpull.png" width="700"> - can be used to supply workers - can build pipelines, producer/consumer --- ## Devices - Queue, Forwarder, Streamer - can serve as a stable point in a system <img src="figures/socket_queue.png" width="700"> --- ## GNU Radio zmqblocks - sink blocks (C++) - source blocks (C++) - "probe_manager" to get signals into Python/PyQt GUI (Python implementation + C++ sink) - "rpc_manager" to send control signals (Python implementation) --- ## Connecting flowgraphs - two flowgraphs can be connected using a sink and a source block - synchronization using the req/rep pattern - source sends request with noutput_items - sink sends message with items, #items <= noutput_items - it is simple here, just memcpy items to zeromq message - hack: build a loop in a flowgraph using a source and sink ;) <center><img src="figures/connected_flowgraphs_simple.svg" width="250"></center> --- ## Polling - zmq send() and recv() functions are usually blocking - use polling to avoid blocking the flowgraph - can be a little bit tricky to figure out zmq::pollitem_t itemsin[] = { { *d_socket, 0, ZMQ_POLLIN, 0 } }; zmq::poll (&itemsin[0], 1, 0); // If we got a reply, process if (itemsin[0].revents & ZMQ_POLLIN) { // Receive data zmq::message_t reply; d_socket->recv(&reply); // Copy to ouput buffer and return memcpy(out, (void *)reply.data(), reply.size()); return reply.size()/d_itemsize; } --- ## PROBE API - connect a zmq pubsub sink to the block you want to monitor self.zmq_probe = zmqblocks.sink_pubsub(gr.sizeof_float, "tcp://*:5556") - add this to your Python GUI # ZeroMQ probe_manager = zmqblocks.probe_manager() probe_manager.add_socket("tcp://localhost:5556", 'float32', self.plot_data) def plot_data(self,samples): [...] - basically creates a watcher thread that unpacks sample data and hands it over to callback function --- ## PROBE API - use a timer to update the plot, e.g. in PyQt update_timer = Qt.QTimer() self.connect(update_timer, QtCore.SIGNAL("timeout()"), probe_manager.watcher) update_timer.start(30) --- ## RPC API - add this to your Python app to receive RPCs rpc_manager = zmqblocks.rpc_manager() rpc_manager.set_reply_socket("tcp://*:6666") rpc_manager.add_interface("start_fg",self.start) rpc_manager.start_watcher() - to be able to send requests add rpc_manager = zmqblocks.rpc_manager() rpc_manager.set_request_socket("tcp://localhost:6666") - send a request rpc_mganager.request("start_fg") rpc_mgr_server.request("set_k",gain) --- ## RPC API - uses GNU Radio pmt's to serialize arguments - watcher will regularly poll for incoming RPC requests - deserializes arguments and calls the interface callback function accordingly --- ## Example <img src="figures/connected_flowgraphs.png" width="700"> --- ## Demos --- ## Conclusion - gr-zmqblocks really useful for remote GUIs ## To Do - implement OFDM feedback loop as a real proof of concept - PyBOMBS recipe - more experiments with buffer sizes --- ## Thanks for your attention Sourcecode: [https://github.com/iohannez/gr-zmqblocks](https://github.com/iohannez/gr-zmqblocks) Slides created using [remark](http://github.com/gnab/remark).