23 May · BCV

Meetings

Link to page

Verify Synchronization and Integrity of bag files

1) Check Bag File Info:

rosbag info S1S1_2024-05-17-merged.bag

path: S1S1_2024-05-17-19-20-31_0.bag version: 2.0 duration: 9:44s (584s) start: May 17 2024 19:20:31.58 (1715970031.58) end: May 17 2024 19:30:15.79 (1715970615.79) size: 154.9 GB messages: 194119 compression: none [10943/10943 chunks]

types:

bcv/EEGData ds4_driver/Status

ds4_driver/Feedback geometry_msgs/PoseStamped geometry_msgs/PoseWithCovarianceStamped geometry_msgs/Twist geometry_msgs/TwistStamped nav_msgs/Odometry nav_msgs/Path sensor_msgs/CameraInfo sensor_msgs/Illuminance sensor_msgs/Image sensor_msgs/Imu sensor_msgs/NavSatFix std_msgs/Float32MultiArray tf2_msgs/TFMessage zed_interfaces/PosTrackStatus

topics: /cmd_vel/joystick 20058 msgs : geometry_msgs/TwistStamped /managed/joy 13988 msgs : geometry_msgs/Twist /odom 13160 msgs : nav_msgs/Odometry /processed_image 6503 msgs : sensor_msgs/Image /robot_status 13156 msgs : std_msgs/Float32MultiArray

/set_feedback 2 msgs : ds4_driver/Feedback /status 54676 msgs : ds4_driver/Status /tf_static 2 msgs : tf2_msgs/TFMessage (2 connections) /yoctopuce/fix 969 msgs : sensor_msgs/NavSatFix /yoctopuce/imu 1011 msgs : sensor_msgs/Imu /yoctopuce/light 1007 msgs : sensor_msgs/Illuminance /zed/zed_node/left/camera_info 12346 msgs : sensor_msgs/CameraInfo /zed/zed_node/left/image_rect_color 2643 msgs : sensor_msgs/Image /zed/zed_node/odom 6141 msgs : nav_msgs/Odometry /zed/zed_node/path_map 875 msgs : nav_msgs/Path /zed/zed_node/path_odom 894 msgs : nav_msgs/Path /zed/zed_node/pose 6147 msgs : geometry_msgs/PoseStamped /zed/zed_node/pose/status 12286 msgs : zed_interfaces/PosTrackStatus /zed/zed_node/pose_with_covariance 6153 msgs : geometry_msgs/PoseWithCovarianceStamped /zed/zed_node/right/camera_info 12328 msgs : sensor_msgs/CameraInfo /zed/zed_node/right/image_rect_color 1796 msgs : sensor_msgs/Image eeg_data 7978 msgs : bcv/EEGData

2) Check for Dropped Messages:

Topic: /cmd_vel/joystick, Frequency: 34.33 Hz Topic: /odom, Frequency: 22.53 Hz Topic: /managed/joy, Frequency: 24.10 Hz Topic: /status, Frequency: 93.59 Hz Topic: /robot_status, Frequency: 22.52 Hz Topic: /tf_static, Frequency: 3708.49 Hz Topic: /yoctopuce/fix, Frequency: 1.66 Hz Topic: /yoctopuce/light, Frequency: 1.72 Hz Topic: /yoctopuce/imu, Frequency: 1.73 Hz Topic: /zed/zed_node/path_odom, Frequency: 1.53 Hz Topic: /zed/zed_node/path_map, Frequency: 1.50 Hz Topic: /zed/zed_node/left/camera_info, Frequency: 21.13 Hz Topic: /zed/zed_node/pose, Frequency: 10.52 Hz Topic: /zed/zed_node/odom, Frequency: 10.51 Hz Topic: /processed_image, Frequency: 11.13 Hz Topic: /zed/zed_node/pose_with_covariance, Frequency: 10.53 Hz Topic: /zed/zed_node/pose/status, Frequency: 21.03 Hz Topic: /zed/zed_node/left/image_rect_color, Frequency: 4.52 Hz Topic: eeg_data, Frequency: 13.66 Hz Topic: /zed/zed_node/right/camera_info, Frequency: 21.11 Hz Topic: /zed/zed_node/right/image_rect_color, Frequency: 3.08 Hz Topic: /set_feedback, Frequency: 0.97 Hz

Comparison Table for Dropped Messages and Frequencies

Comparison Table for Dropped Messages and Frequencies

Topic (Optimal Frequency)

Frequency (Hz)

Dropped Messages

Dropped Percentage

/cmd_vel/joystick (N/A)

34.33

N/A

/odom (N/A)

22.53

N/A

/managed/joy (N/A)

24.1

N/A

/status (N/A)

93.59

N/A

/robot_status (N/A)

22.52

N/A

/tf_static (N/A)

3,708.49

N/A

/yoctopuce/fix (10 Hz)

1.66

4873

83.41%

/yoctopuce/light (4 Hz)

1.72

1329

56.87%

/yoctopuce/imu (100 Hz)

1.73

57409

98.26999999999999%

/zed/zed_node/path_odom (N/A)

1.53

N/A

/zed/zed_node/path_map (N/A)

1.5

N/A

/zed/zed_node/left/camera_info (15 Hz)

21.13

/zed/zed_node/pose (N/A)

10.52

N/A

/zed/zed_node/odom (N/A)

10.51

N/A

/processed_image (N/A)

11.13

N/A

/zed/zed_node/pose_with_covariance (N/A)

10.53

N/A

/zed/zed_node/pose/status (N/A)

21.03

N/A

/zed/zed_node/left/image_rect_color (15 Hz)

4.52

6120

69.84%

eeg_data (125 Hz)

13.66

65047

89.06999999999999%

/zed/zed_node/right/camera_info (15 Hz)

21.11

/zed/zed_node/right/image_rect_color (15 Hz)

3.08

6967

79.5%

/set_feedback (N/A)

0.97

N/A

There are no rows in this table

⁠

Comparison Table of Topic Frequencies of splitted and merged files

Comparison Table of Topic Frequencies of splitted and merged files

Topic

bag_merged

bag_0

bag_1

bag_2

bag_3

/cmd_vel/joystick

34.33 Hz

34.84 Hz

33.18 Hz

35.15 Hz

34.06 Hz

/odom

22.53 Hz

22.42 Hz

22.21 Hz

22.91 Hz

23.55 Hz

/managed/joy

24.10 Hz

26.45 Hz

26.01 Hz

19.39 Hz

31.17 Hz

/status

93.59 Hz

93.06 Hz

92.78 Hz

94.91 Hz

95.11 Hz

/robot_status

22.52 Hz

22.53 Hz

22.26 Hz

22.78 Hz

23.19 Hz

/tf_static

3708.49 Hz

/yoctopuce/fix

1.66 Hz

1.64 Hz

1.57 Hz

1.76 Hz

1.65 Hz

/yoctopuce/light

1.72 Hz

1.71 Hz

1.67 Hz

1.78 Hz

1.73 Hz

/yoctopuce/imu

1.73 Hz

1.72 Hz

1.63 Hz

1.82 Hz

/zed/zed_node/path_odom

1.53 Hz

1.55 Hz

1.51 Hz

1.53 Hz

1.56 Hz

/zed/zed_node/path_map

1.50 Hz

1.53 Hz

1.49 Hz

1.46 Hz

1.61 Hz

/zed/zed_node/left/camera_info

21.13 Hz

20.60 Hz

21.01 Hz

21.87 Hz

21.19 Hz

/zed/zed_node/pose

10.52 Hz

10.31 Hz

10.52 Hz

10.78 Hz

10.44 Hz

/zed/zed_node/odom

10.51 Hz

10.29 Hz

10.47 Hz

10.82 Hz

10.33 Hz

/processed_image

11.13 Hz

11.06 Hz

11.05 Hz

11.31 Hz

11.18 Hz

/zed/zed_node/pose_with_covariance

10.53 Hz

10.34 Hz

10.49 Hz

10.80 Hz

10.43 Hz

/zed/zed_node/pose/status

21.03 Hz

20.54 Hz

21.07 Hz

21.55 Hz

21.11 Hz

/zed/zed_node/left/image_rect_color

4.52 Hz

4.09 Hz

4.32 Hz

5.11 Hz

5.33 Hz

eeg_data

13.66 Hz

13.98 Hz

13.04 Hz

13.98 Hz

13.81 Hz

/zed/zed_node/right/camera_info

21.11 Hz

20.68 Hz

21.00 Hz

21.72 Hz

21.21 Hz

/zed/zed_node/right/image_rect_color

3.08 Hz

3.66 Hz

2.70 Hz

2.86 Hz

3.14 Hz

/set_feedback

0.97 Hz

There are no rows in this table

⁠

Extract Data from ROS Topics

1) Extract EEG Data:

rostopic echo -b merged.bag -p /eeg_data > eeg_data.txt

⁠

eeg_data.txt

3.8 MB

⁠

⁠

2) Extract Button Press Labels:

rostopic echo -b S1S1_2024-05-17-merged.bag -p /cmd_vel/joystick > joystick_data.txt

⁠

joystick_data.txt

1.8 MB

⁠

⁠

labeled_joystick_data.txt

1.9 MB

⁠

⁠

3) Extract Processed Images:

script: extract_processed_images.py

video: processed_image.avi

4) Extract Zed Left and Right Images:

script: extract_zed_left_and_right_images.py

video: left_image.avi and right_image.avi

Filter and Visualize EEG Data

1) Filter EEG Data:

⁠

filtered_eeg_data.txt

3.9 MB

⁠

⁠

2) Visualize filtered data:

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

⁠

Prepare Data for Deep Learning

1) Clean filtered eeg data including eeg channel data + timestamps in CSV

2) Merge clean data with labeled joystick

3) load data data = pd.read_csv('/media/capture/ssd1_2tb/labeled_eeg_data.csv')

4) Split Data into Train and Test Sets

Build and Train Deep Learning Model

1) Define the Model:

class EEGNet(nn.Module):

def __init__(self):

super(EEGNet, self).__init__()

self.fc1 = nn.Linear(X_train.shape[1], 128)

self.fc2 = nn.Linear(128, 64)

self.fc3 = nn.Linear(64, len(data['label'].unique())) # Number of classes

def forward(self, x):

x = F.relu(self.fc1(x))

x = F.relu(self.fc2(x))

x = self.fc3(x)

return x

# Define loss function and optimizer

loss function: nn.CrossEntropyLoss()

optimizer: optim.Adam(model.parameters(), lr=0.001)

2) Train the Model:

number of epochs: 20

3) Evaluate the Model:

Accuracy of the network on the test set: 56.02%

Accuracy of the network on the test set: 59.96%

Performance of experiment:

rostopic hz /eeg_data (headset) average rate: 128.314 min: 0.005s max: 0.029s std dev: 0.00382s window: 127 average rate: 127.123 min: 0.005s max: 0.029s std dev: 0.00334s window: 254 average rate: 126.129 min: 0.004s max: 0.142s std dev: 0.00758s window: 378

rostopic hz /managed/joy (controller) average rate: 121.978 min: 0.001s max: 0.025s std dev: 0.00423s window: 75 average rate: 121.287 min: 0.001s max: 0.041s std dev: 0.00455s window: 196 average rate: 122.217 min: 0.001s max: 0.041s std dev: 0.00422s window: 320

rostopic hz /status (rover) average rate: 121.771 min: 0.002s max: 0.017s std dev: 0.00227s window: 122 average rate: 121.908 min: 0.001s max: 0.031s std dev: 0.00257s window: 244 average rate: 122.927 min: 0.001s max: 0.031s std dev: 0.00233s window: 369

rostopic hz /zed/zed_node/left/image_rect_color (camera) average rate: 6.763 min: 0.137s max: 0.155s std dev: 0.00704s window: 6 average rate: 6.900 min: 0.137s max: 0.155s std dev: 0.00555s window: 13 average rate: 6.819 min: 0.137s max: 0.167s std dev: 0.00703s window: 19

rostopic hz /processed_image (stimuli sys) average rate: 30.699 min: 0.015s max: 0.035s std dev: 0.00382s window: 27 average rate: 30.283 min: 0.015s max: 0.036s std dev: 0.00281s window: 57 average rate: 30.126 min: 0.015s max: 0.036s std dev: 0.00237s window: 86

Before starting expremeient:

perf3 -c 192.168.0.222 Connecting to host 192.168.0.222, port 5201 [ 5] local 192.168.0.192 port 45024 connected to 192.168.0.222 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 18.8 MBytes 157 Mbits/sec 0 759 KBytes [ 5] 1.00-2.00 sec 17.5 MBytes 147 Mbits/sec 0 800 KBytes [ 5] 2.00-3.00 sec 16.2 MBytes 136 Mbits/sec 0 901 KBytes [ 5] 3.00-4.00 sec 15.0 MBytes 126 Mbits/sec 0 946 KBytes [ 5] 4.00-5.00 sec 16.2 MBytes 136 Mbits/sec 0 946 KBytes [ 5] 5.00-6.00 sec 17.5 MBytes 147 Mbits/sec 0 946 KBytes [ 5] 6.00-7.00 sec 15.0 MBytes 126 Mbits/sec 0 997 KBytes [ 5] 7.00-8.00 sec 17.5 MBytes 147 Mbits/sec 0 1000 KBytes [ 5] 8.00-9.00 sec 16.2 MBytes 136 Mbits/sec 0 1000 KBytes [ 5] 9.00-10.00 sec 15.0 MBytes 126 Mbits/sec 0 1000 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 165 MBytes 138 Mbits/sec 0 sender [ 5] 0.00-10.03 sec 162 MBytes 136 Mbits/sec receiver

iperf Done.

⁠

During expremeient:

iperf3 -c 192.168.0.222 Connecting to host 192.168.0.222, port 5201 [ 5] local 192.168.0.192 port 57628 connected to 192.168.0.222 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.40 MBytes 11.7 Mbits/sec 0 120 KBytes [ 5] 1.00-2.00 sec 764 KBytes 6.26 Mbits/sec 0 120 KBytes [ 5] 2.00-3.00 sec 1018 KBytes 8.34 Mbits/sec 0 120 KBytes [ 5] 3.00-4.00 sec 764 KBytes 6.26 Mbits/sec 0 120 KBytes [ 5] 4.00-5.00 sec 1018 KBytes 8.34 Mbits/sec 0 120 KBytes [ 5] 5.00-6.00 sec 1018 KBytes 8.34 Mbits/sec 0 120 KBytes [ 5] 6.00-7.00 sec 1018 KBytes 8.34 Mbits/sec 0 120 KBytes [ 5] 7.00-8.00 sec 636 KBytes 5.21 Mbits/sec 0 126 KBytes [ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 5 91.9 KBytes [ 5] 9.00-10.01 sec 318 KBytes 2.58 Mbits/sec 4 70.7 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.01 sec 7.80 MBytes 6.54 Mbits/sec 9 sender [ 5] 0.00-10.93 sec 7.25 MBytes 5.56 Mbits/sec receiver

iperf Done.

⁠

After finishing expremeient: iperf3 -c 192.168.0.222 Connecting to host 192.168.0.222, port 5201 [ 5] local 192.168.0.192 port 40106 connected to 192.168.0.222 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 1.10 MBytes 9.25 Mbits/sec 0 113 KBytes [ 5] 1.00-2.00 sec 1018 KBytes 8.34 Mbits/sec 0 113 KBytes [ 5] 2.00-3.00 sec 764 KBytes 6.26 Mbits/sec 0 113 KBytes [ 5] 3.00-4.00 sec 764 KBytes 6.25 Mbits/sec 0 113 KBytes [ 5] 4.00-5.00 sec 764 KBytes 6.26 Mbits/sec 0 113 KBytes [ 5] 5.00-6.00 sec 764 KBytes 6.26 Mbits/sec 0 113 KBytes [ 5] 6.00-7.00 sec 764 KBytes 6.26 Mbits/sec 0 113 KBytes [ 5] 7.00-8.00 sec 764 KBytes 6.26 Mbits/sec 0 113 KBytes [ 5] 8.00-9.00 sec 764 KBytes 6.26 Mbits/sec 0 113 KBytes [ 5] 9.00-10.00 sec 764 KBytes 6.26 Mbits/sec 0 113 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 8.06 MBytes 6.76 Mbits/sec 0 sender [ 5] 0.00-10.15 sec 7.62 MBytes 6.30 Mbits/sec receiver

iperf Done.

Want to print your doc?
This is not the way.

Try clicking the ⋯ next to your doc name or using a keyboard shortcut (

CtrlP

) instead.