top of page
  • Andrew Singletary

Robot Safety: Tools and Resources to Help You Build More Reliable Software

Updated: Oct 25, 2023

Building safe and reliable software is hard. Building safe and reliable robots is even harder.


Moreover, just developing bug-free software is not sufficient to ensure the safety of your robot. It’s fundamental with any robotic development endeavor to worry about the quality of the software developed, and that’s not something that can be achieved after the fact. Softwares are built in layers, and are only as reliable as their weakest link, so quality has to be infused in from the very beginning. Accumulating technical debt can be a startup killer – see Argo AI, for example.


The good news is there exist a lot of tools and resources to help developers build quality robotic software – and most of them free. Not only is it a good time investment to learn and set up these tools to manage technical debt long-term – but it’s an extra pair of (bionic) eyes that can be a real time-saver in the short term, by helping you avoid a lot of potentially lengthy bug hunts.

Languages

When choosing your programming language, there are three most common options: Python, Rust, and C++.

Python

  • Rapid development

  • Easy debugging

  • Great resources for ML

  • Bad performance

  • Relies on interpreter

Rust

  • Memory safety

  • Thread safety

  • Performance

  • Immature ecosystem (limited libraries and legacy code)

  • Verbose syntax

  • Steep learning curve

C++

  • Performance

  • Mature ecosystem (libraries and legacy code)

  • Control over resources

  • Not (inherently) thread safe

  • Not (inherently) memory safe

  • Steep learning curve

Overall, we find C++ to be the best balance between safety, performance, and existing ecosystem. However, the inherent safety and similar performance of Rust would make it a more suitable option for robotics development when the ecosystem grows.


Package manager


Package managers help you control dependency trees, have repeatable build environments and ensure ABI compatibility between packages.


The two we use and recommend are:

  • Conan

  • vcpkg


Code IDE


Ditch your outdated notepad++ and jump onto vscode, it has many extensions to help you develop clean robotics code, regardless of your chosen language.


Notable extensions for c++ dev:

  • C/C++

  • Clang-format

  • Clangd

  • CMake Tools

  • Error Lens

  • Test Explorer UI

  • Todo Tree

Notable extensions for ROS dev:

  • Msg Language Suppor

  • ROS

  • URDF

  • Yaml

Notable extensions for embedded dev:

  • Platform IO IDE

  • Arduino


Continuous Integration (CI)


All git repo providers provide some CI with it. Make heavy use of it, it’s (mostly) free extra code reviewers and QA engineers.



Code linting


Code linting prevents errors by catching common mistakes beforehand, maintains consistency in coding styles, enhances readability by suggesting improvements, facilitates collaboration through smoother code reviews, and ultimately improves efficiency by reducing manual review and debugging time.


Best options include:

  • Clang-format

  • Cpplint

  • Flint++

  • include-what-you-use

Unit testing framework


Unit testing ensures reliable code behavior, aids in early bug detection, guides design decisions, supports code documentation, and promotes modular and maintainable code.

  • GTest

  • Catch2

  • Parasoft

  • Boost Test Library

  • boost-ext/ut

Why write tests yourself when someone has already done it for you : SolidSand


Static Analysis


Static analysis identifies complex issues early, enforces coding standards, improves code quality, detects security vulnerabilities, and enhances overall software reliability.


Paid:

  • Coverity Scan

  • PVS studio

  • cppdepend

Free:

  • Cppcheck

  • Clang-tidy

  • clang-sa

Meta:

  • codechecker

Compiler flags:

  • Clang: set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pedantic -Wall -Wshadow -Wextra -Werror -Wnon-virtual-dtor -Wno-long-long -Wcast-align -Wchar-subscripts -Wall -Wpointer-arith -Wformat-security -Woverloaded-virtual -fno-check-new -fno-common -Wconversion -Wextra-semi -Wshadow-all")

  • GCC: set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -pedantic -Wall -Wshadow -Wextra -Werror -Wnon-virtual-dtor -Wno-long-long -Wcast-align -Wchar-subscripts -Wall -Wpointer-arith -Wformat-security -Woverloaded-virtual -fno-check-new -fno-common -Wconversion")


Dynamic analysis and debugging


While static analysis can be great, it’s unlikely to catch every issue you’ll encounter. For this, you need some tools for dynamic analysis and debugging.


Debugging options:

Note: many of these debugging options (and more) interface well with vscode

Compiler flags:

  • -D_LIBCPP_DEBUG

  • -D_GLIBCXX_DEBUG

  • -D_GLIBCXX_DEBUG_PEDANTIC

  • -D_GLIBCXX_SANITIZE_VECTOR


Logging and visualization


Proper logging and visualization provide insights into runtime behavior, aid in debugging and troubleshooting, offer performance monitoring, support data-driven decision-making, and enhance overall system understanding and optimization.


Tools for logging:

  • Spdlog

  • Glog

  • Easyloggingpp

  • Easylogging++

  • rosbag

Tools for visualization:

  • Grafana

  • Foxglove

  • webviz


Code Coverage


Code coverage tools help assess testing completeness, identify untested code paths, guide test creation, enhance code quality, and provide valuable insights into the effectiveness of testing efforts.


Benchmarking


Benchmarking measures performance, identifies bottlenecks, guides optimization efforts, supports informed hardware/software choices, and ensures the delivery of high-performing and efficient software solutions.

  • perf | speedscope

  • Heapstack

  • Strace

  • Dr Memory


Documentation


Documentation is critical. Don’t wait until you’ve forgotten why you wrote it the way you did. Two options we like are:


Coding guidelines


As mentioned previously, C++ is not inherently safe. These coding guidelines exist to ensure the safety of code developed from them.

  • AUTOSAR C++ 2014

  • CERT C++ 2016

  • MISRA-C++ 2008 (2023 coming out soon),

  • C++ Core Guidelines

  • High Integrity C++

  • JSF AV C++

  • JPL 10 rules (https://web.eecs.umich.edu/~imarkov/10rules.pdf)


Operating systems


Your software needs something to run on! Here are some of the most popular real-time (and non real-time) operating systems.

  • Linux

  • preempt-RT

  • Xenomai

  • Free-rtos

  • QNX

  • Vxworks


Robot safety: complex but essential


Merely developing bug-free software isn't enough to ensure a robot's safety. The software's quality is paramount, and this quality should be integrated from the onset of development. We hope our resources list is helpful as you endeavor to ensure the very highest standards or robot safety.


Here at 3Laws, we offer an AI supervisor for autonomous and intelligent robotic systems, ensuring safety without compromising performance. We separate safety from the autonomy stack, and deploy real-time safety analytics, robust collision avoidance software, and technology that's robust to uncertainties in modeling and perception.


Good luck in your mission to ensure that robotic systems function optimally in real-world scenarios!


126 views0 comments

Comments


bottom of page