Sean Johnson Status Report April 25

What I Personally Accomplished This Week

This week I focused on finalizing the navigation system for our demo. I debugged remaining map stability issues to minimize drift during extended navigation sessions. I also worked with the team to plan the logistics of our final demonstration, including defining the test environment layout, determining demo scenario sequences, and identifying what equipment we’ll need for the 15×15 foot test area. Additionally, I contributed to developing content for the final poster.

Progress Status

Progress is on track. All deliverables are progressing as planned for final submission.

Deliverables for Next Week 

Next week I will complete the final poster with all navigation system content and test results. We will conduct the final demonstration. All project components will be complete and ready for final submission.

Sean Johnson’s Status Report for 4/18

What I Personally Accomplished This Week

This week I focused on fine tuning navigation system parameters to improve performance and reliability. I adjusted multiple Nav2 configuration values including obstacle inflation radius in the costmap, local and global path planning refresh rates, and motion filter parameters in Cartographer. I also calibrated motor duty cycle trims to compensate for asymmetry at different speeds, ensuring consistent velocity control across the operating range.

I debugged a critical coordinate frame issue in the OTOS odometry publisher where the sensor’s 90-degree mounting orientation caused X and Y position data to be swapped.I identified and corrected the issue by swapping the X/Y assignments in the odometry publisher code. I also adjusted covariance values to balance trust between odometry and LiDAR scan matching.

I conducted navigation testing including timed distance runs to verify velocity accuracy, localization tests to measure position error, and extended SLAM sessions to check map stability. I also created the final presentation PowerPoint.

Progress Status

Progress is on track with the project timeline. The navigation subsystem is functional with ongoing parameter optimization. 

Deliverables for Next Week 

Next week I will finalize navigation parameter tuning to minimize drift and improve path following accuracy. We will conduct integrated system tests combining computer vision detection outputs with navigation costmap updates to validate human aware path planning. I will also perform the remaining real-world integration tests with team members simulating various scenarios (fallen person obstacle, running detection, multi-person navigation).

New Tools and Knowledge Acquired

Throughout this project, I needed to learn several new tools and frameworks. The most significant was the ROS2 navigation stack, including Cartographer SLAM for mapping and Nav2 for path planning. I had no prior experience with SLAM algorithms or costmap based navigation, so understanding how to configure occupancy grids, inflation layers, and path planners required substantial learning through online guides and youtubes.

I also relied heavily on trial and error testing. For example, tuning the odometry covariance values required running multiple SLAM sessions, observing drift patterns, then systematically adjusting values and re-testing. When debugging, I used a systematic experimental approach: I would make a hypothesis, design a quick test, observe the result, then iterate. This strategy was more effective than randomly trying solutions. I also searched GitHub issues and ROS Answers forums when encountering specific errors, which often provided insights from others who solved similar problems. 

This project also required working with the Raspberry Pi 5 beyond what I have done in the past. More significantly, this was my first experience integrating a large multi-subsystem. Issues could stem from hardware connections, software interfaces, or coordinate transformations. I learned to use isolation testing and found importance in clear interface definitions between modules developed by different team members.

Sean Johnson’s Status Report for 4/4/26

What I Personally Accomplished This Week

This week I focused on demo preparation and hardware refinement. I tuned the SLAM mapping parameters to improve map quality and stability for the demonstration. I assembled the updated chassis with new 3D printed components and secured connections using hot glue to ensure mechanical stability during testing. I also prepared the demo presentation and validated that the navigation operates reliably for the demonstration scenario.

Progress Status

Progress is on track. The navigation system is stable and the hardware assembly is complete with improved mechanical mounting.

Deliverables for Next Week

Next week I will fine tune the mapping and path planning parameters to improve navigation performance. We will also begin integrating computer vision outputs into the navigation system to enable human detection.

Verification:

For LiDAR verification, I am testing scan rate consistency and range accuracy by monitoring the RPLidar A1M8 output on the scan to verify it maintains 5-10 Hz publication frequency and validating distance measurements by placing known obstacles at measured distances to confirm its within 5cm tolerance. Paul and I performed verification testing for physical movement and battery runtime by running movement scripts at different speeds. For battery testing, we ran the complete system doing navigation loops at approximately 1 m/s for over 45 minutes repeatedly, exceeding our 30 minute runtime goal.

Sean Johnson Status Report for 3/28/26

Sean’s Status Report for March 28, 2026

What I Personally Accomplished This Week

This week I focused on physical system assembly and integration testing. I built and organized the chassis, mounting the motor drivers, Raspberry Pi, KR260, and wiring all electrical connections. I worked with the team on testing the integrated navigation stack with the motor control system and debugging issues with the RPLidar initialization timing. I also contributed to planning our interim demo, where we aim to demonstrate functional autonomous navigation with basic computer vision human detection.

Progress Status

Progress is on track for the interim demonstration. The chassis is assembled and ready for testing, and the navigation stack is operational. Integration work continues as planned.

Deliverables for Next Week

Integrate for human detection data from the KR260 and validate end to end communication between vision and navigation systems. Testing of the complete system on the assembled chassis to ensure reliability. 

Sean Johnson Status Report for 3/21/26

What I Personally Accomplished This Week

This week I focused on validating the navigation with actual robot motion by acquiring and integrating an Elegoo Smart Robot Car as a testing platform. I purchased and assembled the Elegoo chassis to enable immediate testing without waiting for our final mechanical design. The assembly involved mounting components, connecting the TB6612FNG motor driver, and wiring the Arduino UNO for motor control. 

Worked on connecting Nav2’s velocity commands to the Arduino controlled motors. This required writing a ROS2 node that converts Nav2’s /cmd_vel messages into serial commands sent to the Arduino over USB. Testing revealed an issue where Nav2 publishes only angular velocity commands with no forward linear velocity. I traced this to a coordinate frame mismatch between the odometry data and Nav2’s expectations. I also completed the required ethics module assignment. I also worked with the team on soldering connections for the motor driver power lines and sensor wiring, ensuring reliable electrical connections for sustained testing sessions.

Progress Status

Progress remains on track with the project timeline. The Week 7 deliverable for motor integration has been achieved with the Elegoo test platform responding to Nav2 commands.

Deliverables for Next Week 

I will begin implementing the behavioral finite state machine for operational mode transitions and establish the TCP socket receiver on the Raspberry Pi to accept human detection data from the KR260. These deliverables will complete the navigation subsystem and enable full system integration testing in Week 8.

Sean Johnson Status Report for 3/14/26

What I Personally Accomplished This Week

This week I focused on integrating the complete navigation stack with SLAM and beginning end to end testing of autonomous path planning. The major accomplishment was successfully configuring Cartographer SLAM to handle faster robot motion, which was causing map distortion issues with our previous slam_toolbox implementation.

I created and tuned a custom Cartographer configuration file that increases the scan matching search window and adjusts motion filtering parameters to prevent tracking failures during rapid movements. The configuration specifies appropriate parameters for our RPLidar A1M8 sensor including 12m max range, and 5cm map resolution. This resolved the map distortion problem where the entire map would shift and warp when the robot moved quickly.

In coordination with Paul and Justin, I worked on initial motor setup and constructed a temporary testing car to validate the navigation with actual movement. We assembled a basic wheeled chassis and connected the motors to the Raspberry Pi. This temporary test vehicle allowed us to verify that velocity commands from Nav2 can successfully control physical motors, though we encountered some challenges with motor response calibration that will need further tuning. Having a mobile platform, even in this preliminary form, was essential for testing whether our SLAM and navigation algorithms perform correctly during actual motion rather than just simulation.

I also completed the required ethics module assignment for the course.

Progress Status

My progress remains on track with the project timeline. The Week 6 milestone (March 14) called for closed-loop navigation capability, and while the navigation planning pipeline is now functional and generating valid paths and velocity commands, full closed loop execution awaits motor controller integration from Paul. The remaining work involves connecting these velocity commands to actual motor control and integrating the human detection data from Justin’s vision pipeline to enable the human avoidance behaviors specified in our design.

Deliverables for Next Week

Next week I will begin implementing the behavioral finite state machine that manages transitions between our five operational modes (NORMAL, CROWD_AVOIDING, DENSE_CROWD, EMERGENCY_FALL, EMERGENCY_RUNNING). This FSM will subscribe to human detection data and velocity commands to determine appropriate robot behavior. Once Paul completes the motor controller, I will integrate it with Nav2’s velocity commands to achieve actual autonomous navigation. Finally, I will conduct comprehensive testing of the navigation stack in various hallway scenarios to validate path planning performance and identify any parameter tuning needed for smooth operation. 

Team Status Report 3/14/26

Significant Risks and Mitigation

    The primary risk is integration timing between the three parallel development tracks. The vision pipeline on the KR260, navigation stack on the Raspberry Pi, and motor control system must converge for end to end testing. If any subsystem falls behind, full system validation will be delayed, potentially impacting our Week 9 Hamerschlag Hall demonstration. We are managing this by establishing clear interface specifications including TCP socket communication for human detections, ROS2 topic structures for velocity commands, and serial command protocols for motor control. Each subsystem is being developed with stub nodes and mock data that allow independent testing without dependencies on other components.

    Hardware reliability presents ongoing challenges. In our unit testing so far, we are happy with our odometry drift mitigation, but are worried about it for longer routes. We experienced intermittent I2C communication issues with the OTOS sensor that required power cycling to resolve. We are also maintaining backup components for critical sensors where budget allows.

    Computational performance on the Raspberry Pi 5 remains a concern. Running Cartographer SLAM, Nav2 navigation stack, odometry processing, and eventual human detection data fusion simultaneously could exceed the Pi’s capabilities. Initial testing shows acceptable performance with SLAM and Nav2, but we have not yet stressed the system with the full sensor suite and motor control running concurrently. Our contingency plan involves optimizing SLAM update rates, reducing Nav2 planning frequency, and potentially offloading some processing to the KR260 if performance degradation occurs.

    The switch from slam_toolbox to Cartographer introduced a new risk around loop closure and map consistency during long navigation sessions. Cartographer’s submap approach is more robust to fast motion but requires careful parameter tuning to prevent map drift over time. We are monitoring this through extended mapping sessions and will adjust loop closure parameters if necessary.

Design Changes

     We made a significant change from slam_toolbox to Cartographer SLAM this week. The original slam_toolbox implementation experienced severe map distortion when the robot moved faster than walking pace. The entire map would shift and warp, making navigation impossible. Cartographer’s scan matching algorithm with larger search windows and motion prediction from odometry handles rapid movements more robustly. This change required creating a custom Lua configuration file with parameters tuned for our RPLidar A1M8 sensor and modifying launch files to use Cartographer instead of slam_toolbox. The time cost was approximately one day of configuration and testing, but the benefit is substantial improvement in mapping stability during actual robot motion.

     Another design decision was abandoning Gazebo simulation in favor of exclusive real hardware testing. We discovered the Raspberry Pi 5 GPU only supports OpenGL 3.2, while Gazebo Harmonic requires OpenGL 3.3 or higher. Rather than purchasing additional hardware for simulation, we elected to conduct all testing on the physical platform. This actually accelerates our progress by forcing earlier confrontation with real-world sensor noise, timing issues, and hardware integration challenges that simulation would not reveal.

     A major design shift was moving from UART to TCP socket communication between the KR260 and Raspberry Pi. The original design used UART serial communication for transmitting human detection bounding boxes, but this proved unreliable for the data rates required when multiple humans are detected simultaneously. TCP over WiFi provides higher bandwidth, built-in error checking, and simpler debugging. This change required setting up static IP addressing on the Pi via netplan and implementing socket-based communication on both the KR260 transmit side and the Pi receive side.

      We constructed a temporary testing platform using an Elegoo Smart Robot Car V4 chassis to validate the navigation stack with actual movement before the final mechanical design is complete. This allows the navigation and motor control subsystems to develop in parallel with the mechanical assembly. The Elegoo platform uses the same TB6612FNG motor driver as our final design. We debugged the full Nav2-to-motor-control pipeline where Nav2 publishes cmd_vel, a Python ROS2 bridge node computes differential drive kinematics, and sends serial commands over USB. The project remains on schedule. Week 6 milestone for closed-loop navigation has been achieved with the full Nav2 stack running and motors responding to velocity commands. Week 7 priorities include implementing the behavioral FSM, establishing TCP communication for human detections from the KR260, and tuning the NMS post-processing pipeline. Week 8 focuses on system integration testing, and Week 9 remains on track for Hamerschlag Hall demonstration.

Team Status Report for 3/7/26

Our most major risk is being behind on physical movement, which delays our physical tests and integration of computer vision/localization and movement systems.

We did not receive the correct batteries that we had ordered. They had the capacity we need, but with a maximum continuous current of 3A compared to the 6A that was specified. This would make the batteries unable to handle the surge in current draw from our motors during stalling or startup, which presents a serious physical safety risk. We have already addressed this by adjusting our testing and procuring a different battery and power delivery system that should fit our needs. Portions of physical controls testing were done by using power supplies on campus, which allows for us to test basics such as starting, speed controls and stopping. Given the inability to do a large portion of movement testing, Paul has been temporarily reassigned to assist Justin and Sean. Until the new parts arrive, we have been using the most advanced of testing for our Lidar and computer vision: dragging the chassis with a wire:

This week the majority of work focused on the software navigation stack running on the Raspberry Pi 5. The goal was to get from a bare ROS2 Jazzy install to a fully functional autonomous navigation system with human interaction protocols, and most of that was achieved by the end of week with hardware integration remaining.

SLAM and TF Tree Debugging

The first major thing was getting GoogleCartographer working with the RPLiDAR A1 on ROS2 Jazzy. TF tree configuration required significant debugging. Cartographer needs a specific chain of coordinate transforms to function: map -> odom -> base_link -> laser_frame. Each of these transforms has to come from a specific source, and if any of them are missing or have timestamp issues, Cartographer silently drops all incoming laser scans and produces no map.

The first issue was that the LaserScan messages were being dropped by the ROS2 MessageFilter before they even reached Cartographer. This happens when the transform from the scan frame to the fixed frame does not exist at the timestamp of the scan message. The fix was ensuring robot_state_publisher was running with the correct URDF describing the static joint between base_link and laser_frame, so that transform was always available.

The second issue was t/*-he robot_state_publisher was broadcasting the base_link to laser_frame transform with timestamps that were 507 seconds in the past, derived from stale joint state messages. Cartographer would reject any transform older than a few seconds, so even though the transform existed it was effectively invisible. This was fixed by setting ignore_timestamp: True in robot_state_publisher’s parameters, forcing it to always publish with the current time regardless of the joint state timestamp.

The third issue came after integrating the OTOS odometry node. Cartographer has a parameter called published_frame which controls what frame it publishes its pose estimate into. When odometry is enabled, this must be set to odom, not base_link. Setting it to base_link caused Cartographer to publish a map->base_link transform at the same time the OTOS node was publishing an odom->base_link transform, creating a direct conflict in the TF tree that made the robot’s position undefined. Once published_frame was correctly set to odom the two systems no longer conflicted.

OTOS Odometry Integration

The PAA5160E1 is an optical flow sensor combined with an IMU that tracks the robot’s position and orientation by watching the floor surface, similar to how an optical mouse works. Integrating it into ROS2 required writing a custom node that reads from the sensor over I2C, converts the native units (inches and degrees) to ROS standard units (meters and radians), and publishes both a nav_msgs/Odometry message on the /odom topic and a TF broadcast of the odom->base_link transform. A few things mattered a lot for stability. The QoS profile for the odometry publisher needed to be set to BEST_EFFORT to match what Cartographer expects on its odometry subscriber, using RELIABLE caused messages to be silently dropped. The publishing rate also needed to be as high as possible, so the node uses a tight while loop rather than a ROS2 timer, which gave more consistent TF timestamp continuity and reduced Cartographer’s transform lookup failures.

The OTOS is sensitive to mounting. During early testing the sensor was held by hand while moving around the room and the resulting maps were unusable, every small vibration or tilt registered as robot movement, causing Cartographer to accumulate errors rapidly and produce false loop closures where the map would suddenly jump to a completely wrong position. Mounting it firmly on a flat surface dramatically improved map quality.

Motor Controller

The motor controller node was written for the two Pololu VNH5019 H-bridge motor driver boards that will drive the Hiwonder DC motors in a differential drive configuration. This is entirely WIP until we get the actual motor drivers.

Design changes:

Due to the aformentioned battery issue, we have transitioned to a different power delivery system based on what we were able to quikcly procure. Our new battery is a bienno 12afaf, which is a 12V battery with a 20A max continuous current. This also requires new equipment such as a power delivery block and adapters for the powerpole system.

Part A: Written by Paul Wright

Hospitals are well-known for their huge cost struggles and that need high uptime on equipment. With the prohibitive cost of healthcare, an expensive product can decrease the total amount of care provided  if it drives up cost more than the benefit it offers.  Our product solution addresses this through a low cost and the use of available, off-the-shelf parts. These parts have large amounts of publicly available documentation and open-source projects that use them, allowing for easier repairability and modification.  A decrease in the need for a specialist is of special benefit to rural or disconnected areas, as they face a greater time and monetary penalty than urban centers when they use them. Every minute that a device is not working when expected harms people by affecting logistics and lowering the standard of care. Any device with moving parts will inevitably need maintenance and/or repair, so making it more accessible should increase uptime and reduce costs.

Hospital populations are generally under high stress and have a large variation in the physical condition of patients. For any high-stress environment, assistive products should allay concerns, not introduce new ones. Our product solution avoids becoming a new source of stress through proper detection of people and avoiding them. To properly understand what it needs to avoid, the robot needs to be able to properly distinguish between soft (movable) and hard (nonmoving) obstacles and know where it is. Classification of obstacles is achieved through proper tuning of mapping parameters, while localization uses both lidar and odometry to more accurately find it’s location even with rapid changes in the environment. Ultimately, patients and employees should not have to concern themselves with the vehicle. 

Part B: Cultural Factors, by Sean

Our autonomous hospital delivery robot is designed with careful consideration of the cultural factors important to healthcare environments. Hospital culture places paramount importance on patient safety, dignity, and privacy, which directly influences our design decisions. The emergency detection protocols of fall detection and running detection reflect the cultural values of healthcare institutions where staff are expected to immediately respond to patient emergencies. Our behavioral finite state machine implements culturally appropriate responses such as stopping and alerting staff rather than attempting autonomous intervention, respecting the cultural norm that medical emergencies require human healthcare professional judgment. Additionally, hospitals have behavioral codes and regulations including HIPAA privacy requirements, which our system respects by not storing or transmitting images, only processing them in real time for detection. Also, cultural acceptance of autonomous robots varies significantly across patient populations, like how elderly patients may be less comfortable with robotic systems. Our design addresses this by ensuring the robot operates predictably, maintaining appropriate clearance distances from all individuals.

Part C: Environmental Factors, by Justin

Our delivery robot addresses environmental factors through energy efficient design and sustainable operational practices. By utilizing rechargeable lithium ion batteries rather than disposable power sources, the system minimizes battery waste. The robot’s energy consumption is optimized through efficient path planning algorithms that minimize unnecessary travel distance, and the 0.5 m/s maximum velocity specification balances delivery efficiency with power consumption. Our component selection prioritizes longevity and repairability with a modular design and off the shelf components (Raspberry Pi, RPLidar, motors) that allows for individual component replacement. This approach reduces electronic waste compared to proprietary integrated systems. Acoustic pollution is minimized through quiet motor operation and scarce use of audio alerts only during emergency conditions. While the manufacturing of electronic components has environmental costs, the system’s ability to potentially operate continuously for years with minimal maintenance provides a net environmental benefit over its lifecycle.

Sean Johnson’s Status Report for 3/7/2026

What I Personally Accomplished This Week

This week I worked along with Justin on the core software infrastructure setup for the  navigation system, establishing the foundation for autonomous navigation and SLAM. I began by configuring the Raspberry Pi 5 platform with Ubuntu 24.04 LTS and ROS2 Jazzy. The choice of Ubuntu 24.04 over 22.04 was necessary to support ROS2 Jazzy, which provides better compatibility with our navigation stack. The major hardware integration accomplishment was successfully connecting and configuring the RPLidar A1M8 sensor. I installed the rplidar_ros package for ROS2 Jazzy and configured the sensor parameters. The LiDAR is now publishing scan data at stable rates. I also established the proper tree structure connecting map to odom to base_link to laser_frame, which is essential for SLAM operation.

For SLAM implementation, I installed and configured slam_toolbox for ROS2 Jazzy, then switched to google cartographer. This required debugging lifecycle node activation requirements, which are specific to the Jazzy distribution and differ from previous ROS2 versions. I created configuration files targeting 5cm resolution mapping to meet our design specifications. I also made progress on the navigation by installing Nav2 and all required plugins including the A* planner and various costmap layers. Initial integration testing between SLAM and Nav2 components has begun, although full closed loop navigation is still in progress.

One thing I found out this week was the Raspberry Pi 5 GPU limitation. While investigating Gazebo simulation for testing purposes, I found that the RPi 5 only supports OpenGL 3.2, but Gazebo Harmonic requires OpenGL 3.3+. This led to the technical decision to prioritize real hardware testing over simulation. 

Progress Status

My progress is on track with the project timeline, achieved the Week 5 milestone. The foundational infrastructure is in place, but significant work is needed to complete the navigation system including full Nav2 path planning testing, custom human detection costmap layer implementation, and motor controller integration for actual navigation execution. Despite this, the current pace aligns with our project schedule.

Deliverables for Next Week

Next week I will focus with Justin on completing the SLAM/Nav2 integration to achieve stable autonomous path planning with real time map updates. A key milestone will be demonstrating initial closed loop navigation in our controlled lab environment. This means the robot should be able to accept a goal position, plan a path, and provide velocity commands that will eventually drive the motors (once the motor controller integration is complete).

I will begin developing the custom costmap layer for human detection as specified in Equation 3 of our design report. This layer will apply exponential cost inflation around detected humans with parameters w_h = 10.0 and σ = 1.0 meter, ensuring the robot maintains appropriate clearance from pedestrians.

Sean Johnson Status Report for 2/21/26

What did you personally accomplish this week on the project?

This week I focused on investigating alternative localization approaches after our professor had skepticism about relying solely on the optical flow sensor (PAA5160) for accurate navigation. We received feedback that odometry localization may not be reliable enough for our accuracy requirements. The main alternative I explored was marker systems, specifically ArUco markers, AprilTags, and QR codes that could be placed at known locations throughout the building hallway. The idea is that our existing camera could detect these markers and use them to reset or correct position estimates during navigation, compensating for any drift accumulated by the optical flow sensor.

I evaluated several sensor fusion architectures that could combine multiple localization inputs. The most promising approach uses the odometry sensor for continuous tracking between landmarks and visual fiducial markers for periodic position correction. This hybrid system would leverage the strengths of each component while mitigating their individual weaknesses. For implementation I researched how the OpenCV ArUco detection library could integrate with our existing camera hardware and CV pipeline. A hybrid approach using optical odometry for continuous tracking with periodic visual landmark corrections could achieve our position error requirements while being significantly more robust than odometry alone. ArUco markers can be detected in real time with our YOLOv3-tiny human detection without creating a significant performance bottleneck.

Is your progress on schedule or behind?

According to our Gantt chart, Task 3.1 should be complete by the end of Week 2. While I’ve made progress researching the optical sensor and understanding its capabilities, the professor’s feedback revealed that we need a different solution. This puts me a few days behind on finalizing our localization approach since we now need to integrate additional components we hadn’t initially budgeted time for. Our team will make a decision on our approach very soon, whether that’s pure odometry or odometry with visual landmarks.

What deliverables do you hope to complete in the next week?

My primary deliverable is getting the optical flow sensor operational with working code. I need to write the interface code to communicate with the PAA5160 sensor and pull X/Y displacement readings. This will be implemented in Python interfacing through the Arduino. Once I have raw sensor data coming through, I’ll implement a basic position tracking algorithm that integrates the displacement values over time to estimate the robot’s current position.