WiMi Hologram Cloud Inc., a leading global Hologram Augmented Reality (AR) Technology provider, announced its development of a humanoid robot control system based on a head-mounted display (HMD) controlled by a brain-computer interface (BCI).This system via steady-state visual evoked potentials (SSVEP) and allows interaction with the environment and humans. The technology provides real-time feedback through the robot‘s embedded camera, which integrates stimulus feedback into the HMD display.
WiMi‘s researchers took control of the robot and tested its performance by using this new interaction in an experiment. Testers were asked to navigate the robot to a specific location to perform a task. This test was based on visual SLAM feedback, which provides navigation instructions through images of the environment captured by a camera.
Steady-state visual evoked potentials (SSVEP) are used as control signals, and the user’s EEG signals are captured by an EEG signal acquisition device, and real-time feedback from the robot is displayed using a head-mounted display. For navigation instructions, the researchers embedded an embedded camera in the robot and combined its real-time feedback with the SSVEP signals on the head-mounted display to form a form of interaction. The researchers used a visual SLAM (Simultaneous Localization and Mapping) algorithm to implement the navigation instructions.
CIO INFLUENCE: CIO Influence Interview with Herb Kelsey, Federal CTO at Dell Technologies
The control system of the head-mounted display BCI consists of several components: signal acquisition device, head-mounted display, humanoid robot, embedded camera, control algorithm, etc. The steps and results of WiMi‘s BCI control system implementation are as follows:
EEG signal acquisition and processing:
The WiMi researchers first used a set of EEG signal interaction platforms for capturing the user’s EEG signals and processing those signals. The platform consists of a multi-channel amplifier, electrode caps, and data acquisition software to capture and store the user’s EEG signals in real-time. In processing the EEG signals, an efficient method is used, which is the extraction of SSVEP by frequency analysis of the signals. This method presents light flashes of a specific frequency to the user, and the user’s EEG signal will resonate at this frequency, thus enabling the extraction of control signals.
The researchers used an EEG signal acquisition device to capture the user’s EEG signals, a head-mounted display to show real-time feedback from the robot, and an embedded camera to provide navigation instructions. The control algorithm uses a visual SLAM algorithm to implement the navigation instructions, which models the environment and implements the navigation instructions from images of the environment captured by the robot‘s camera.
CIO INFLUENCE: Top Challenges for CTOs in 2023
Head-mounted display applications:
To provide feedback on the control signals, the researchers used a head-mounted display, which shows real-time images and states displayed through a built-in monitor.The head-mounted display utilizes high-resolution display technology to provide a more realistic experience of the virtual environment.
In addition, to better provide feedback on control signals, the researchers combined the display with the robot‘s built-in camera. With the robot‘s camera capturing images of the environment and displaying them on the head-mounted display, users can visualize the robot‘s state and environmental information more intuitively. Specifically, the user is required to gaze at a specific frequency of light stimuli on the head-mounted display, which causes the user’s brain to emit a specific SSVEP signal. Once the signal is captured and processed, a control algorithm can determine the user’s intent based on the frequency and amplitude of the signal, such as moving the robot to turn left or right. The embedded camera captures the viewpoint of the robot in real-time, providing an image of the environment as well as feedback on the current position of the robot.
Navigation implementation:
Using the SLAM algorithm, an image of the robot‘s environment is captured by the robot‘s built-in camera and converted into a model of the robot‘s environment. The algorithm is also able to estimate the robot‘s position and provide navigation instructions to the user. The user can control the direction of the robot‘s movement through SSVEP signals, while the head-mounted display shows the robot‘s status and environmental information to provide more intuitive feedback to the user.
In addition, the researchers conducted an experimental evaluation of the control system to assess its performance in terms of control accuracy and interaction experience. The experimental results show that the control system can provide precise control signals as well as an immersive interactive experience, providing users with a novel way to control their robots.
CIO INFLUENCE: General Data Protection Regulation (GDPR) Anniversary
Experimental evaluation:
To evaluate the performance of the system, the researchers conducted a series of experiments. The experiments included a user using the system for a robot navigation task and a control experiment using a traditional remote control for the task. The results of the experiments showed that users were able to perform navigation tasks more accurately and get better interaction when using the system.
The researchers conducted experiments to evaluate the control system, including performance in terms of control accuracy and interaction experience. In the experiment, participants were required to control the robot to walk to a specific location to complete a task in a simulated environment. The experimental results show that the control system can provide precise control signals with an average control accuracy of 98.1%. Meanwhile, users evaluated the system as a very good interaction experience, and considered it a very natural and intuitive way of interaction.
WiMi‘s head-mounted display (HMD) for robot control via a brain-computer interface (BCI) demonstrates a new way of interaction, providing a more natural and intuitive way of control. Experimental results show that the control system can provide precise control signals. This control method has the potential to be used in many scenarios that require precise control, such as medical, educational, and entertainment fields. Subsequently, the control system can be combined with other sensor technologies, such as voice and gesture, to provide more diverse control. This navigation-assisted scheme provides users with a novel interaction method, which can improve the robot‘s operation efficiency and interaction experience.
CIO INFLUENCE: Nextira Selected by Ansys Technology Partner Program to Support Customers Implementing Ansys Gateway Powered by AWS
[To share your insights with us, please write to sghosh@martechseries.com]
