Category: Team Status Report

Significant Risks

This week we were able to successfully implement usage of the IMU along with the rest of the localization system, so we are able to constantly display the user’s rotation data along with the user’s position. We were also able to test the localization system in a larger area somewhat similar to the space in the Wiegand Gym to make sure the tag would be able to read the distances between it and each of the anchors. The testing was mostly positive: although the range of the tag and anchor is approximately 34 meters (connectivity beyond this range is still possible, though insufficiently consistent for our localization system), in practice, practice, the range of the device seems to be a little more inconsistent at a range of around 25 meters. The firmware of the tag attempts to switch to connecting to a different anchor if any packets are dropped between the tag and the anchor, and this may happen at a greater frequency at ranges greater than 25 meters, which can cause the anchor connections to become less stable. However, this testing does show that placing anchors in a denser fashion improves the stability of its connections in the network. In conclusion, this week we were able to resolve the outstanding deliverables we had promised to implement as a part of our project.

Changes to System Design

We incorporated the IMU into our localization system this week to display the rotation/heading of the user on our map. This was largely successful; hence there were no system design changes required.

Schedule

This next week our team will be primarily working on the final deliverables for the project. We also aim to do some further testing (optimistically in the Wiegand Gym itself if we can enter at a time where the basketball activity on the courts is not especially high). We will also work on our presentation for demo day.

Progress:

The video below displays the capabilities of the authentication system, with the user being directed to create a new account, as well as being able to edit some aspects of their account and sign out using a profile page.

Link

This Week’s Prompt:

Unit tests:

1. Range of the anchors and tag. This was done to ensure that the maximum range of communication is sufficiently large enough so that installation of the system would not be excessively expensive. Our target goal was 25 meters. We measured the furthest distance in a hallway such that our tag could still read distances from an anchor and found that this distance was around 34 meters. This met the test, and so we did not need to change our anchor placement strategy.
2. Distance measurement accuracy between an anchor and a tag. How much does distance measurement actually differ between an anchor and a tag? Our target goal was 0.23 meters (the rationale of this number comes from the maximum change in distance in a typical 25×2 meter hallway to cause the localization error from multilateration to differ by 1 meter). The actual observed distance error as measured by taking an average of 10 readings is 0.15 meters, which passes the test. These numbers reveal the localization system is theoretically capable of highly accurate localization.
3. Localization Accuracy of the localization system. We compared the average predicted position of the localization system with the actual position we were standing in a hallway and find that the average misprediction distance was approximately 0.2 meters (1 measurement taken in 2-meter increments over a 20 meter stretch of hallway). We tested the difference between using a trilateration and multilateration algorithm on localization accuracy. Overall, both are competitive. However, due to the behavior of the DWM1001’s to sometimes fluctuate from inside of a hallway, we find that the multilateration can improve accuracy by approximately  0.1 meters. These numbers meet our goal of <1 meter accuracy and are sufficient in accurately tracking a student.
4. Localization Precision: We want to ensure the predicted user’s position does not sporadically jump on the screen. We stood in a location and measured the frequency/distance of jumps we would see in the localization system. We notice the maximum fluctuations were approximately 2.1 meters, which was larger than the 0.5 meter goal. Because of this, we designed a filter for the final position estimate, that uses the change in estimated position over time to calculate our velocity, and rejects data points that would imply a velocity higher than some maximum (which we specified to be 2m/s).
5. Heading Accuracy: We want to make sure the difference in the angle of the user’s estimated orientation aligned with that of reality. To test this, we walked along the hallway, rotating around, before stopping at parallel angles with the hallway (in real life) before comparing the result to that of what was seen on the browser. We took the difference between that angle and the actual angle. The average result over 15 trials reveal the average difference in angle is 20 degrees. This aligns with our goal of 20 degrees.
6. Battery life of tag: We tested the battery life of the system by running the raspberry pi with the localization system constantly running. Our battery life lasts for at least 10 hours, which beats our desired goal of 4 hours.
7. Position Update Latency: We measure the latency of our multilateration and nelder-mead algorithm and find it is on average 20 ms, which is sufficiently less than the goal of 500 ms to make sure that the latency of the algorithm is fast enough for fast update frequency.
8. Distance update frequency: We measure the frequency at which the tag can get values from anchors and find this value is 10.1 Hz. This reveals our localization system can easily provide frequent updates for the user’s position (as long as other factors are not bottlenecking the process).
9. Tag to Webapp Latency: We find that the latency of communication between the user’s Raspberry Pi and the server is on average around 225 ms. However, this best case condition relied on the server processing many packets at once (which in some cases it cannot keep up) so we ended up switching to using Websockets, which decreased our average time to 67 ms. Due to our navigation system not requiring incredibly fast latency to avoid disorientation, we find that this is sufficiently fast enough to provide timely updates. Particularly, we learn that this latency makes the rotation update frequency run on the RPi in a fluid manner.
10. Total Latency: We measure the total latency between user movement (e.g. past a door) and it being shown on the web application to have an average delay of around 0.84 seconds. We believe this sufficiently in preventing a user from getting lost.
11. Navigation Algorithm: We benchmark the speed up the navigation algorithm and find that it produces results in an average of 125 ms, which is sufficiently fast to provide the user with instructions when they are using the application.
12. User Experience: We surveyed 4 of our potential clients (students whom were unfamiliar with a building) to use our application to navigate to a specific room. We asked them to fill a quick survey to gather some of their feedback, using the qualitative feedback to improve some aspects of the webserver’s user interface. The user’s quantitative rating of the clarity and helpfulness of the directions was a 4.5/5.0.

 

Significant Risks

This week we were able to work on development of the localization system on the Raspberry Pi 4. Through this process, we were able to fix a major issue the speed of our distance acquisition speed on the tag device (overall improving the distance acquisition rate from anchors from approximately 1 Hz to 10.1 Hz), resulting in significantly faster update speeds.

We spent time this week focusing on large scale testing in third floor of ReH. The ~62 meter long hallways in this building are significantly longer than some of the previous tests we have run in the 26 meter hallways of HH A level, which are a closer approximation of what we might find when demoing in a larger environment such as the Wiegand Gym (with dimensions of approximately 32×40 meters). This testing process resulted in us finding that the firmware of the DWM1001 tag devices supports a maximum of communicating between 4 anchors at time–this in itself is not a problem, because multilateration using 4 anchor distances seems to result in relatively accurate results. However, we found that the tag device seems to have difficulties in swapping between anchors to communicate with. From the documentation, it appears that it is supposed to connect with anchors based on their “quadrant” (e.g. connecting to the nearest anchors in each quadrant). We found our tag device to have some difficulties in finding the closest anchors, sometimes preferring to “stick” to a set of anchors that are still in range, yet further away. Our main goal for the following week is to address this issue.

There are several ideas we have began to float around. One is to configure the initial locations of the anchors, which could help the tag better identify which anchors are available for it to connect to at a given time. There are strategies we could implement to reset the connections between the anchors and the tag to force the tag device to reconnect with the closest anchors in its range.

Changes to System Design

This week we were able to resolve an issue with the tag device not being able to update it’s position frequently enough (from 1Hz to 10.1Hz, as described earlier). Hence, we have chosen to remove the IMU position estimation we had planned earlier to include. With our higher update frequency, we find it is no longer necessary to have a reliance on an IMU to affect the user’s position as the “ground truth” of the UWB localization system is more frequently updated.

Schedule

This next week we will be working on improving the localization system so that it adapts better to regions with more than 4 anchors. This is expected to be a team effort, as it takes a considerable man effort to map out buildings, set up the anchor positions in the correct area, as well as do further configuration with the tag/webserver. We also hope to continue working on the final deliverables required, such as the poster.

Progress

The following is a video of the localization system and navigation system working in tandem in HHA level, with the localization system running on the Raspberry Pi. The localization system runs at a higher frequency than before and then the navigation system is in a state such that it is able to provide directions, as well as distance/time estimations for the travel time.

Video Link

The following is another video of a “simulated” version of the navigation system working, displaying more details including the system providing new directions to the user.

Video Link

Significant Risks

This week we spent a significant amount of time testing the localization system for our interim demo. We were ultimately able to demonstrate the accuracy of the system, though the update frequency of the system running on the Raspberry Pi is insufficient. Going forward, one of our main goals is to continue testing the Raspberry Pi to make sure that the updates are more frequent. We have already made several attempts to refactor the code to use alternatives to our gradient descent with PyTorch, and have seen some improvements in performance. However, we have noticed this may have come with the tradeoff of accuracy, and we need to continue doing further optimizations. We would also like to try offloading some of the computation onto the webserver and see if it is powerful enough to do some of the computation required.

Changes to System Design

Originally we assumed an accelerometer/gyroscope would be sufficient for getting the user’s orientation, though Ifeanyi’s testing showed that it would be more accurate to also use a magnetometer as well. We are hoping that this chip may provide more useful readings than our previous chip, assisting with IMU orientation and any location prediction using that chip. Hence, acquired a new imu that had magnetometer features as well. The rest of the system design is unchanged.

Schedule

This week we worked on setting up the interim demo and additional improvements to the localization system, as well as starting on the navigation system. Next week Weelie will be working on improving the navigation system, doing more testing on the RPi and improving the latency of the update frequencies. Ifeanyi will be working on the RPi, programming the IMU and getting rotation data. Jeff will be continuing to work on the navigation system so that it can be used to provide the user with navigation directions.

Progress

Here is a video of the initial portion of our navigation system. The video shows a demo of the system working, such that the webserver is running on my local machine and I have a script sending in “dummy data” for the user’s position. Entering a new room using the search bar sends the asynchronous request to the server, which responds with the path the user should take, and is drawn on the user’s map.

Video Link

Validation

There are a number of validation tests we need to do to ensure the system satisfies the use case requirements of our clients and properly functions.

Firstly, we need to ensure that the localization system is accurate and provides timely updates. We would like the accuracy of the localization system to be on average within one meter inside of a building. We need to measure this by going to a position in a hallway and comparing the result of the localization system to the user’s actual position, before taking the difference of the distances. This accuracy is sufficient for preventing the user from becoming lost in the building, as well as decreasing the amount of fluctuations in the user’s position.

Additionally, we want the position updates to happen smoothly at a high frequency of greater than 2 Hz, as well as not have too much delay between physical movements and updates on the server. These tests will require more coordination between all team members, because these tests are dependent on how long it takes for both the localization system and user front end to run its algorithms. As long we are able to establish accurate timestamps for communication start/end times, we should be able to obtain these measurements. Also, we need to make sure that the lagging time is less than 1 second. We will verify this by taking 2 points, one is the start and the other is the end. Then, we should walk from the starting point towards the end point. We should make sure that after we reach the end point, our position on the map will update accordingly within 1 second. We should repeat this test with different walking speed to cover different situations.

Furthermore, we need to test our navigation algorithms. We can do this by benchmarking it on some example graphs with known shortest paths, to see if our solution recommends these same paths. It really is as simple as making sure the function always returns the optimal path as it is supposed to. Additionally, if we make the nodes of our graph consist only of space on our map that is walkable (that is, no walls or other fixtures), then we are assured that our returned paths will always follow journeys the are indeed walkable by the user.

Lastly, we need to validate our directions and our user experience. To do this, we need to test our app several times with different starting locations and destinations to ensure that the directions we are given in real time always line up with the path recommended by A*. Then, for the user experience, we will survey a few participants to see if the app works according to its use case statement. That is, given only the purpose of the product, can a new user pick up the device and navigate successfully to their location? We will also survey them about the quality of the directions and the user experience for possible improvements to the interface.

Significant Risks

The most significant risk we currently face is that we have finally gotten our localization system with at least 3 anchors up and running, and yet, we are seeing less than desirable results with regards to localization frequency (not quite < 2 Hz)  and position accuracy (the tag’s distance measurements seem to have variance of up to +/- 0.75 meters). To improve the latency, we are implementing various optimization strategies such as running a thread in the background to process the data, and are already seeing slight speedup. For position accuracy, we are currently tinkering with slightly different methods of calculating the position using the 3 distances. We are also implementing smoothing that averages at least the last 3 measurements. We might also see accuracy gains if we use more anchors.

Changes to System Design

We were able to successfully set up a webserver and send localization updates to the webserver. Because of this, there was no need to make any updates to the system design. 

Schedule

This week we were able to further test the localization system. We were able to have the tag find the positions of at least five anchors simultaneously, and we also found that the maximum distance we could communicate between a tag and an anchor appeared to be approximately 35 meters, which is above the 25 meter benchmark we were aiming for in our Design Report. 

Next week is the runup to our midpoint demo; hence, we would like to continue improving our localization accuracy, augment our user orientation using the IMU data, and continue making improvements to the throughput between our webserver and the client by testing websockets. 

An updated schedule with our current progress can be found here.

Significant Risks

This week we resolved the issue of forming a network with our DWM1001 development boards due to Weelie working on a multithreading task to communicate between multiple anchors. The volume of anchors a tag can communicate has not been verified yet, as we have not had the chance to test it with a larger number of anchors. We still need to find if there is an upper limit of anchors we can communicate with. Suppose we are only able to communicate with a limited number of anchors at once. In that case, we will need to tweak the tag’s routine so it initiates communication with a smaller subset of the anchors at once.

Finally, we still need to try setting up the “gateway” with the RPi4 reading from the DWM1001 board’s serial to make the RPi suitable for creating our connection to the internet. If our USB connection doesn’t work, we can try using SPI instead, which should work based on the datasheet.

Changes to System Design

This week was spent focused on the initial design of the network of ultrawideband transceivers. Our initial testing seemed positive; hence, we did not make any changes to the system design.

Schedule

This week was spent working on more of the initial setup for the DWM1001 development boards and making progress on the webapp.

We hope to spend next week working toward our midpoint demo. We would like to be able to localize our device within a room or a table, along with creating a connection to the webserver.

The following schedule includes the current progress we have made on our respective tasks.

18500 Gantt Chart 3-16

Progress

A video demo of some progress on the webapp’s “floor view” can be found here.

A video demo of progress made on the tag/anchor communication can be found here.

 

Significant Risks

This week was spent mostly working on the design review report; hence, our significant risks are still similar to those discussed in our previous report. Namely, there could be difficulties in setting up an ultrawideband network of DWM1001 boards communicating amongst each other. We need to ensure we can create this network of devices, as well as find the limit (if there is one) to the number of devices the tag can communicate with at any given time. Also, we need to make sure to set up a Gateway using a RPi. If there are issues in getting the RPi to interface with the DWM1001, we can try to use a ESP32 UWB as our gateway instead.

Changes to System Design

Our research into the design requirements whilst working on the design review report revealed that a DWM10001-Dev board paired with a RPi 4 might be a better solution using the ESP32-UWB module. This is because the RPi 4 offers significantly higher computing power that can run the localization algorithms faster, as well as any processing that needs to be done with the IMU.

We also discussed integrating sensor fusion techniques with the IMU, using a Kalman filter to decide whether to use readings from the localization network or a prediction estimate from the IMU. We think this technique could lead to an improvement in localization and are planning on continuing with it.

Schedule

This week we primarily worked on the design report. We revamped our Gantt chart to better show the work breakdown, as well as adding milestones we would like to hit.

18500 Gantt Chart breakdown

Prompt

Part A was written by Jeff, Part B was written by Ifeanyi, and Part C was written by Weelie.

Part A: Our project is named “Scotty Maps” because we wanted to create an application tailored specifically to the CMU campus. Although our web application will be specifically based around in CMU, it is not to say the project will not have a global impact. We hope to design a localization system such that it is easily adjusted to fit other buildings. By creating a localization cheaper than others, as well as well as developing tooling to help map buildings, it is entirely possible that our project can be used in a myriad of other locations, whether it still be in academic buildings, or any other commercial location in need of a navigation system.

Assuming there is widespread demand for indoor localization systems, our device could provide a massively improved quality of life for people trying to find locations in buildings by making navigation more interactive. It could affect the way navigation “assistance” is currently accomplished in buildings such as shopping malls: instead of needing to hire people to provide directions or using large monitors displaying the floorplans of the building, our product would be able to provide shoppers with a more interactive experience, showing directions and easily leading people to where they need to go.

Part B: Our project has a significant cultural impact for students across campus. This comes from the fact that students design their entire lifestyle around being on time and organized for everything that they take part in on campus. Part of these considerations involves sticking around the parts of campus that they are most familiar and can therefore navigate the most easily. This leads most students to frequent the same spots on campus for the majority of their time at the university. With our device, students can more confidently explore and traverse new parts of the campus that they are not at all familiar with. Another part of student culture is that in preparation for urgent events, such as midterm exams or finals, students have to make out time to visit the venue and learn how to get to it so that they don’t arrive late to the exam on the day. But with our device, there won’t be nearly as much of a time sink in finding a new place on campus that you’ve never been to before. Finally, students have a culture of sometimes wanting to keep to themselves. This may be for all sorts of personal reasons, such as social anxiety. With our device, they wouldn’t have to approach strangers to ask for directions to places when they get lost trying to find their way somewhere.

Part C: Our project used a lot of UWB devices to send and receive signals, and we will use wifi to transfer the location data to the web server. This may bring the concern that the signals may impact people’s health. Although UWB devices has shorter range than WIFI signals, we can get more precise location by using UWB devices than using WIFI signals. There always concerns for WIFI and  phone signals. Some people believe that the radiation will harm human bodies. However, the power of using UWB is actually lower than WIFI signals. And WIFI signals’ power is lower than an iPhone’s power.  Even iPhone’s power radiation will not do harm to humans. Thus, our devices’ radiation will not exceed the limit for human body.

Significant Risks

This week we found we were able to successfully program the DWM1001 development boards as well as measure distances between them.

There is a risk in the fact that there could be difficulties in setting up an ultrawideband network of DWM1001 boards communicating amongst each other. We need to ensure we can create this network of devices, as well as find the limit (if there is one) to the number of devices the tag can communicate with at any given time. We would like to have these figured out next week so we know the limits of the devices relatively early on into the development cycle. If it turns out that there is a limit to the number of devices communicating, our contingencies consist of opting towards utilizing multiple UWB channels for our communication networks.

We also need to make sure that the DWM1001 boards can be connected to the internet so we can communicate data to the user. To do this, we need to make sure to set up a Gateway using a RPi. If there are issues in getting the RPi to interface with the DWM1001, we can try to use a ESP32 UWB as our gateway instead.

Changes to System Design

Because our initial progress with the development of both the web application and the DWM1001-Dev boards seemed to be overall positive, we did not make any changes to the design of the system.

Schedule

We made some slight changes to our schedule to accurately describe our tasks. Some existing tasks were reorganized.

18500 Gantt Chart 2-24

Next week’s goals:

Jeff: Continue development of the web application, making improvements to the building view to add functionality for navigation.

Weelie (also Ifeanyi): Accomplish a full “network test” this week to get unique distances from multiple devices. Is there a limit to how many devices we can talk to?

Ifeanyi (also Weelie): Put in an order form for a RPi. Then see if you can use the RPi to talk to the internet while interfacing with the DWM board. Research a good IMU (or use the one already on the DWM1001 board) to use with the RPI.

Team: work on Design Report.

Web App progress

The most significant risks that could jeopardize the success of the project are difficulties in programming the DWM1001-DEV boards, as well as UWB being less robust than we initially assumed. The DWM1001 development boards use the nRF52832 microcontroller, and programming it with the functionality that we want might be more difficult due to possible lack of support in areas such as maintaining a larger network of devices or accurately calibrating for antenna delays. If these aspects seem to interfere with our project, we can either work on implementing some of the capabilities ourselves or try pivoting to using a different microcontroller with the DWM1001 boards, such as the ATMega328p, which could have more library support. Additionally, UWB could be less robust than initially assumed, providing less accurate readings indoors. The possible contingencies we are thinking of right now are to add more sensor fusion from IMUs, or in the worst case pivoting towards using a different technology altogether.

 

This week we finished flushing out a design of the system using UWB instead of Wi-Fi, and we were able to create an updated block diagram that more precisely reflects our understanding of the design. 

 

Last week we said that we were going to include a new schedule, and this week we spent some time developing it. 

18500 Gantt Chart 2-17

 

Goals for the next week

Put in order forms for all the all the components we want. 

Run initial testing DWM-1001 MDEK boards. Try to find distance between an anchor and a tag using to way ranging time on arrival (TOA) and make sure that it is accurate.

Continue working on setting up the webapp. 

 

Status Report Prompt: A was written by Ifeanyi, B was written by Jeff, and C was written by Weelie. 

 

Part A: With regards to health, safety and welfare, our project relieves anxiety related to being lost in unknown parts of campus. This anxiety can be especially severe when one is alone, as there would be nobody to ask for directions. And even in the presence of others, finding somebody willing to lead you all the way to a class can be hard, and time consuming, much like the alternative solution of wandering the building until the desired room is found by trial and error. In general, our project greatly aids the mental well-being of students by providing convenient access to anywhere they need to go, as well as saving students from the stress of arriving to events late by saving them time in the room-finding process.

 

Part B: One way our project could affect social factors is to help students navigate campus better, which could bolster or provide an incentive for students to explore campus. Some students may often feel stressed when they are tasked with going to a new building, and this device may alleviate some of the negative feelings those students may have. Our mapping software could help students save time looking for classrooms or offices, making their schedules more efficient. 

Our device could lead to privacy concerns for students, as it would consist of placing a tracking device on every student. A bad actor would be able to see the locations of students using the device. It would be important to avoid this from occurring by placing safeguards so that the privacy rights of students are respected. The device could also create dependency of students on the navigational technologies, hindering their abilities to navigate among rooms on campus without it. 

 

Part C: We use 12 different UWB devices for anchors, IMU and esp32 for the tag. The tag itself is not expensive, which may only take 15 dollars. However, combining with UWB devices, since each floor probably needs 12 UWB devices, for a whole building, this could be pretty expensive. For a 3 floor building, the total cost could be over 1,000 dollars. For individual users, the tag itself is easy to buy and for distribution. But for all of the anchors, it’s better for a company or the owner of the building to buy the whole system, and provide services for individual users. Since companies can also charge users for buying their services, depending on different companies, this could save their cost. For individual users, their cost will increase if they need to buy the services from companies, but it should not be as expensive as purchasing a single anchor($33). 

This week we spent time researching the various technologies and algorithms that could be useful in indoor localization and navigation. Overall, our findings reveal that high localization accuracy indoors might be able to be achieved in reality.

The project is still in an early planning phase, hence, we have chosen to pivot from using Wi-Fi to using UWB due to the hopes of being able to achieve higher accuracy and precision in localization. Finally, we also discussed the complexity of the project and are possibly investigating methods that could expand the scope of the project. For example, other localization techniques, such as doing computer vision on the surroundings of the user, could also accurately localize the user. Effecting such sensor fusion of UWB and CV could improve the accuracy of localization.

We are still in the midst of discussing how best to go forward and will create an updated schedule for next week.