Category: Team Status Reports

This week, we spent most of our time working on obtaining testing values for our presentation. We felt that this went successfully since all of our tests passed. Since we have achieved our MVP, we are not too concerned about obstacles that we may face. Our focus is now to get multi-finger functionality working, which will begin with two finger zoom. Additionally, we may try to make a frame to improve the appearance of our project. All updates to our schedule were shown during the presentation.

This week, we connected all edges of the frame and spent most of our time debugging the hardware and software, as well as collecting data for measuring our performance against our use-case requirements. Beginning with the hardware, one issue we encountered was a misprint in the bottom PCBs. We were able to hack around this by soldering wires onto the board to fix the broken connection. Before we started testing our software with the frame, Matt S conducted some final debugging and checks to make sure there were no issues with the hardware.

Once the hardware was ok to go, we attached the frame to Matt S’ laptop and began testing of Matt K’s updated Arduino code, which controls the LED/Photodiode pairs and sends data about which pairs are activated to the python backend via USB serial. We encountered some bugs, but eventually after some squashing we were able to confirm that the control signals and output data were correct.

Afterwards, once we confirmed the Arduino subsystem was well-integrated with the hardware, we began our first true test of our entire system through a python script that used the data sent over by the Arduino to calculate coordinates and send commands to Darwin’s updated Touch Control subsystem. At first, a few bugs had came about due to a small miscommunication about the updated interface, but we were able to quickly fix them. Once we were passed that, we finally got our first glimpse at Touch TrackIR in action! Physical finger taps, holds, and drags on the screen were successfully translated to the expected touch inputs in the OS. We were able to click on links, close tabs, move windows, and scroll through webpages. The smoothness of the experience is acceptable, but we still feel there is room for improvement.  For example, we can reduce latency by increasing baud rate, removing print statements, and by simplifying the structure of the data that is being used in our coordinate calculations to eliminate unnecessary loops.

Overall, thanks to the progress this week, we put ourselves perfectly on schedule. Next week, we plan on constructing a cover for our PCB frame and adding support for double taps to allow us to perform “right clicks”.

This week, much of our focus was on the project demo. While we are still awaiting the decoders for our PCBs, we were able to make use of the test breadboard from earlier in the semester. On the software side, we developed a script to showcase touches on this breadboard to create clicks on the screen. While this was a good way to give an idea of how our project works, this week we must try to finish the functionality of single-finger gestures by using state logic to distinguish between clicks and finger slides. We should also try to minimize glitches that occur from LEDs being blocked for only brief moments in time. On the hardware side for the coming week, all we can do is hope that the decoders finally come in. If they do not, we will likely need to start hacking through-hole versions of the decoders onto the board so we can start debugging appropriately, especially since we have started eating into our slack time.

This week the PCBs finally came in, and we performed a lot of team work since we worked together on soldering the components. However, one issue came up, which is that the wrong decoders had been ordered, so we are now waiting for the new ones to come in so that we can finish assembling the PCBs. The most significant risk right now is the arrival time of the decoders since we cannot move forward with physical testing until these arrive. One mitigation plan that was developed this week was to create more advanced software tests that can simulate physical data. This should help the debugging process once the physical testing begins, and this is important since the physical testing is probably the most time consuming component remaining. No changes were made to the design of the system this week. As far as the schedule, as long as the decoders come in soon, nothing should change. As long as they are delayed, everything else will be pushed back, and the new software testing will fill the gap.

This past week, we have continued integration between subsystems B (Arduino data processing) and C (Python Screen Control Interface). This will allow us to send touch control requests to the OS from the Arduino via USB. Integration between subsystems A and B will begin on a later date, as we still need to finish the PCB and frame. We have been held back on the arrival of our components, but fortunately we finally have them on our hands and are eager to start getting our hardware subsystem ready. Next week, we plan on starting the assembly of subsystem A by dedicating our time to PCB soldering and finalizing frame design, potentially also starting frame construction. Given the slack time we gave ourselves, and our ability to shift focus on the software as we waited for parts, we feel we are currently on schedule with our tasks.

Over the past week, our team has focused on finishing the PCB and software testing. We are falling a little behind schedule on the software side since we still do not have a working concept of the Arudino and Screen Control integration. As we wait for the PCB this week, this should be our main task. Additionally, since the PCB is taking a lot longer than advertised, we are a week behind on our soldering plan. Fortunately, our Gantt chart has ample slack time to work with for our current standing. Our updated schedule will have all of our plans shifted back by a week with the above tasks inserted into this upcoming week.

Our biggest goal right now is to finish all of the PCB development and order the PCBs before spring break so that we can begin soldering and testing with the actual PCB once classes resume. If this is not completed before break, it is going to cost us about two weeks, so we are considering this the biggest risk to our project right now. We plan to have all three group members working on this task next week to ensure that it is completed.

Overall, no major changes have been made to the design. If anything, the biggest change is the choice to move away from the pyFirmata library for the Arduino-Python interface and to the serial library. This change was necessary due to the fact that each photodiode needed 10ms to be read, which will be far too slow for MVP goal. Moving forward, we will test different baud rates with the serial library to see how fast the system can perform.

Right now, the schedule remains the same since we are keeping up with the tasks on our Gantt chart. Overall, we feel that we have been making good progress and our project is on task to be completed successfully.

This week, we tried to do some preliminary, small-scale testing of our idea. We made a small circuit and tested it with an Arduino. A couple of design changes we made involved how we plan to capture Python data. While Matt K initially experimented with controlling the Arduino with Pyfirmata, this proved to be an unreliable method of rapid data collection. Going forward, we will need an implementation that has data collection already programmed onto the Arduino, and simply use Python to receive the data over USB-serial. One other change we made was that, as we discussed as a possibility from last week, we needed to add a MOSFET buffer between the Photodiode and the Mux.

As for schedule changes, the hardware side is moving at the expected pace, and we hope to have PCBs ready to order by the beginning of March. The software side of things is moving okay, although we still haven’t made the for-certain determination that Python will be fast enough. But, from all of our Python tests so far, we have not encountered anything problematic enough to transition to C++.

Overall, we feel that we made significant progress on our project this week. The most important update is that we switched from an analog approach for determining finger position to a digital one. This change affords us several advantages; for one, it eliminates the need for a preliminary simulation, giving us at least an extra week of slack. Additionally, it removes a lot of the complexity of an analog approach, as digital detection will be far more immune to noise. The largest drawback of this approach is that it is more brute-force than the analog, and we will need far more diodes, which could provide some challenges as to how we package the circuit into a limited form factor. However, we think the pros significantly outweigh the cons, as the digital implementation is less risky in the long run. Our block diagram from the initial proposal remains largely unchanged, since our initial idea was also a digital approach.

We have identified two risks that could significantly affect the outcome of the project. The first is that we have limited space on the bottom edge of the laptop to place the hardware. Before switching to the digital approach, this was not a concern since the hardware would have only been on two edges, but it now needs to be on four edges since we are creating a 2D grid. For the laptop we plan to design the product for, there is currently an inch of vertical space below the screen for the hardware to be placed, so we think it is possible that this is sufficient. However, since it is not a guarantee, our current plan would be to build outwards towards the keys if we need more space. With the shift of our schedule, we should have more time to focus on the design of the frame, so this hopefully help mitigate the risk. A second risk we have identified is that the cost of the PCBs may put us over our budget. With our new approach, we have decided that PCBs are probably necessary in order to hold the hardware components in place. We currently do not have a specific estimate of how much this will cost, but we our planning to figure this out in the next week. As a worst case scenario, we have decided that we could have a 2D touch grid on the side of the laptop if we become affected by these risks.

Updated Schedule: