A Real-time, Remotely Operated Vehicle, using a Raspberry Pi and a Virtual Reality Headset

Abstract

The usage of modern information and communication technologies, such as virtual and mixed reality, offers new options for controlling and monitoring IoT devices. For example, Head Mounted Displays (HMDs) are gaining popularity as a tool to enhance user productivity and enjoyment. This development is also related to the recent advancements in computer technology and the decline in the price of that technology: HMDs are now more functional while also being more widely available on the market. This paper presents a two- wheel robot car that can be controlled remotely in real-time using HMD. The remote control is done in Virtual Reality with the help of Unity 3D. The open-source game engine decreases cost and development time. There are separate objects for the steering wheel, transmission, screen, and stop button. Both controllers and the user’s hands can be used as input manipulators. The Oculus headset's external cameras use hand recognition to implement this feature. The Raspberry Pi 4 has three main functions: first is to control DC motors with GPIO pins, second is to send video stream from the camera to HMD and third is to accept control signals from HMD and perform them. The data transfer of the Virtual Reality headset and Remotely Operated Vehicle (ROV) is done through server-client communication. Raspberry plays the role of a server, which is written on the Flask framework of the python programming language. This server works using asynchronous principles and the OpenCV library for working with images. GPIO pins are controlled by the server, and it receives requests as well. VR headset is a client, which is written in C# on the Unity game engine. The device interacts with the server when the user does any action and transfers the video stream to the screen in real-time. The configuration of input systems is done with the help of the official Oculus Software Development Kit. The convenience of new input systems and their inherent advantages and disadvantages are discussed. Full-scale tests and findings on whether the suggested approach is practical for actual offshore activities are described as well. The results reveal that it is easier and less expensive to modify the input layout while maintaining the same message-sending technology without the limitations of a physical control panel for the ROV operator.