RTAS2021 Materials...

... coming soon!

Heterogeneous Real-Time and Visual Servoing Systems


Today's embedded systems (ranging from small mobile phones to large-scale distributed designs) rarely feature only one type of processing resource (e.g., a processor or an FPGA) and only locally attached connections to the outside world (e.g., to sensors and actuators). Instead, multiple - potentially heterogeneous - processing units are interconnected by a communication network to satisfy the requirements of the job in hand. Most importantly, these are reliability, predictability, performance and overall system cost.

Interface and Communication Issues at hand

Besides these computational aspects both design and verification of the communication subsystems (managing the data flow not only on one local processing unit, but also between several distributed nodes) become more and more challenging. Locally, this is caused by continuously increasing data rates and the need for flexible data routing during runtime due to reconfiguration of the processing resources. On system-level, data routing has to be synchronized to global computation planning to enable smooth fail-over in case computation node or communication link outages.

Experimental Setups

To evaluate both functional and temporal aspects of our proposed architectures, we rely on a Magnetic Levitating Sphere as a Representative Use Case. Continuous levitation is achieved via Visual Servoing, i.e., optically detecting the sphere’s current position followed by computing the required actuation signal within the order of milliseconds. The Visual Servoing System (consisting of actuator, sensor and heterogeneous Processing Platform) relies on Gigabit Ethernet as a Communication Backbone. With Ethernet now being widely used in industrial automation and many Visual Servoing Applications requiring similar low-latency Image Acquisition and Processing, our Experimental Setup thus serves as a representative use case for such scenarios

Experiments with a nearly-zero latency sensor (i.e., a light barrier) have shown that stable levitation is possible with sampling rates as low as 125 Hz. With Visual Servoing, however, the increased sensing delay (due to image acquisition and processing) requires compensation via higher rates.

The Magnetic Levitating Sphere in action!

Theses & Internships

Currently, the following thesis topics are available - please get in touch for more details:

  • Development of an HDMI interface on Zynq FPGAs in VHDL

The following theses are already assigned or completed:

Please refer to the RCS theses page for additional topics.


  • Geier, Martin and Pitzl, Florian and Chakraborty, Samarjit: GigE Vision Data Acquisition for Visual Servoing Using SG/DMA Proxying. ACM/IEEE Symposium on Embedded Systems for Real-Time Multimedia (ESTIMedia) (ESTIMedia'16), ACM, 2016, 19--28 mehr… BibTeX Volltext ( DOI ) Volltext (mediaTUM)
  • Geier, Martin and Franzen, Fabian and Chakraborty, Samarjit: Hardware-accelerated Data Acquisition and Authentication for High-speed Video Streams on Future Heterogeneous Automotive Processing Platforms. International Conference on Computer-Aided Design (ICCAD '18), ACM, 2018, 93:1--93:6 mehr… BibTeX Volltext ( DOI ) Volltext (mediaTUM)
  • Geier, Martin and Burghart, Tobias and Hackl, Martin and Chakraborty, Samarjit: In situ Latency Monitoring for Heterogeneous Real-time Systems. International Conference on VLSI Design and International Conference on Embedded Systems (VLSID '19), 2019 mehr… BibTeX Volltext (mediaTUM)
  • Geier, Martin and Faller, Dominik and Brandle, Marian and Chakraborty, Samarjit: Cost-Effective Energy Monitoring of a Zynq-Based Real-Time System Including Dual Gigabit Ethernet. 2019 IEEE 27th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), IEEE, 2019 mehr… BibTeX Volltext ( DOI )


The following individuals are currently working on this project:

  • Martin Geier focuses on the design of network and communication sub-systems, both between two and inside one processing unit(s).