Open-source, copy and modify what you need!
Open-source, kopírujte a upravujte co potřebujete!
Note: Original branch of master's thesis -> switch to Master's Thesis
RM1 is an experimental robotic platform created to automate antigen testing. This project was developed as part of a master's thesis. The aim was to create a functional and modular prototype that is easily modifiable and deployable after debugging. The basic idea is to create a web-based server that communicates with ROS. ROS was used in the work as a simulation and debugging environment, mainly for robot control. The thesis is divided into four main parts:
In this repository there is a Python Flask Server, that takes care of running the kernel on the back-end, one of the curiosities is that the Intel Realsense camera is connected to the server. This is based on the original design, which intended for a local server only and ROS was to be used for robot control. The gripper and force-torque sensor are controled and included in ROS, because the RG2 and Hex-e sensor are connected to the robot.
For a fully functional application, it is necessary to implement both RM1-Server and RM1-ROS in your operating system. So you can run the simulation and see the application in operation. For real-world application, it is advisable to print the enclosed RM1-Gripper models with the fasteners on a 3D printer and assemble them.
During the design, emphasis was put on modularity, for the benefit of the user it is possible to run the server in Docker-Compose or purely in Python, for more information go to the /app directory.
Note: If there will be a problem with poetry in the requriements.txt file there should be a working library configuration.
Supported resolution:
- 1920x1080
- 1280x960
Server software equipment
| Part | Software |
|---|---|
| Conteiner | Docker-Compose |
| Package manager | Poetry |
| Back-end | Python 3 |
| Front-end | JavaScript, CSS, HTML |
| System run | Bash |
- Search patient in database by PIN(Perosnal Identification Number)/Surname or QR code reader with PIN
- Face ID - compare of patient from streaming color image by photo to database
- Face detection - detection of human face landmarks
- Scan face - scan aligned color image to depth and reconstructed 3D point cloud
- Identification of center of nostril - own programmed seg. cnn for nostrils with post processing of centroid
- Manual control of robot - simple UI for control robot joints
- Show point cloud in browser
- Show digital twin in browser
- Get basic data about weather
- Streaming color, depth, infra image
- Streaming basic data from robot
- Generate PDF document base on HTML input
- Brute force test to crack password
- Basic Python tests
- Pick and place, motion to detected center of nostril
- Simulation x Real world control of robot
The solution was tested on a single board computer Nvidia Jetson Xavier Development Kit. This computer is based on ARM architecture while this server was developed on x86 architecture. The alternative of adding Docker Compose for ARM is not currently addressed. The solution has been tested on the Flask development server. Here are some notes on how to get the solution running on ARM:
- Install Python >=3.8
- Do not use Poetry package manager, some libries raise up installation error.
- Install the libraries directly into Python PIP.
- The main problem arises with Intel Realsense. -> It is recommended to build the SDK from source.
Procedure:
- Download librealsense source as .zip: /~https://github.com/IntelRealSense/librealsense/releases/
- After downloading the .zip file, its contents should be extracted so that you have a librealsense folder
- go to the librealsense root directory
$ mkdir build && cd build
$ cmake ../ -DFORCE_RSUSB_BACKEND=ON -DBUILD_PYTHON_BINDINGS:bool=true -DPYTHON_EXECUTABLE=/usr/bin/python3.8
# Note: -DPYTHON_EXECUTABLE=/usr/bin/python3.8 -> path to installed Python
$ make -j4
$ sudo make install
$ export PYTHONPATH=$PYTHONPATH:/home/jetson/Desktop/librealsense-2.50.0/build/wrappers/python
# Note: librealsense-2.50.0 -> depands on version of downloaded version SDK (if you not change name)
$ sudo nano .bashrc
# Note: paste into .bashrc command export PYTHONPATH=$PYTHONPATH:/home/jetson/Desktop/librealsense-2.50.0/build/wrappers/pythonSource: https://couka.de/2020/10/27/jetson-nano-install-librealsense-with-python-bindings/
Click to full resolution
- Author: Martin Juricek
- Designer: Katerina Monsportova
- Supervisor: Roman Parak
If you want to cite please check the header repository.
github.com/Steigner/Robo_Medicinae_I =>