Enes Palaz Status Update

• What did you personally accomplish this week on the project?
  • Worked on optimizing parameters for edge and contour detection to get best visuals for the demo.
  • Worked on documentation of test results for the final report.
  • Worked on testing demo setup with planned white backdrop.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • Everything is on schedule and we are ready for public demos.

• What deliverables do you hope to complete in the next week?
  • We’re done!!!

Jake Zimmer Status Update

• What did you personally accomplish this week on the project?
  • I got the sub board fully working and integrated with the full camera raspberry pi
  • I rewrote the SPI communications part of the dac driver firmware stack to make it much faster. Before we were limited by the Linux SPI implementation to around 70k transfers per second. This meant that we were often struggling to hit our goal of of 12kpps because each point requires 6 transfers. Now, we are able to perform hundreds of thousands of transfers per second.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • On schedule

• What deliverables do you hope to complete in the next week?
  • We’re done!

Eliana Cohen Status Update

• What did you personally accomplish this week on the project?
  • Helped out more with integration
  • Generated some graphs for our test results
  • Tweaked the interpolation values for the laser

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • On Track – We meet our requirements, and we’re fairly satisfied with our final project, though we saw room for improvement through a different raspberry pi GPIO library that may have faster output to our DACs

• What deliverables do you hope to complete in the next week?
  • We’re done!

Team Status Update

• What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 
  • We managed to get a stable setup for the public demo. Hopefully, we will be ready to do a colorful and visual demo on public demo day.

• Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? 
  • Project structure stayed the same but we played with system parameters for ILDA generation and edge detection to get best visuals for demo. This was planned so no structural changes were necessary.

• Provide an updated schedule if changes have occurred. 
  • No changes in schedule.

Enes Palaz Status Update

• What did you personally accomplish this week on the project?

This week, I continued working on connecting double buffer to ILDA conversion and DAC subsystem. Because of the asynchronous nature of all these subsystems debugging takes time. I managed to identify key breaking points and actively working on them. Also this week we encountered another bug caused by OpenCV package that prevents us from setting camera resolution so I am also working on compiling OpenCV manually to our resolution specifications.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

We are not currently behind but I expect integration to take one more week so I moved them forward in our schedule while we continue working on other deliverables in parallel. No extra action needed to catch up since this problem doesn’t affect other hardware and fabrication tasks.

• What deliverables do you hope to complete in the next week?

I am planning to finish debugging the integration of our software submodules and successfully compile OpenCV manually to our specifications.

Jake Zimmer Status Update

• What did you personally accomplish this week on the project?

I acquired the parts and finished assembling the boards for the system. Right now, I have left some parts off because during testing, we found them to not be required. These parts consist of the op amp circuitry and were easily bypassed. If we decide in the future to use the opamp, it can easily be added in. More testing needs to be done to the board during the integration phase.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I was not behind schedule but the order of some tasks changed. I have updated the schedule to reflect them.

• What deliverables do you hope to complete in the next week?

By the end of next week, the board hardware should be all done and the physical enclosure should be mocked up.

Eliana Cohen Status Update

• What did you personally accomplish this week on the project?
  • Was away for most of the week for job interviews, so wasn’t able to accomplish anything.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I’m behind at the moment to catch up on other classwork, but I had anticipated I’d be unavailable for this week and so the time was pre-budgeted. I’ll pick up on debugging code and integrating hardware next week.

• What deliverables do you hope to complete in the next week?
  • Help setup and test Safety Subsystem
  • Debug ILDA/DAC libraries depending on what still needs to be completed by mid-week

Team Status Update

• What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

The current risk is the set of problems that we are having during integration of our software submodules. For this not to be a serious delay, both Enes and Eliana are actively working on getting this running. We believe we can solve these issues like we solved previous performance and ILDA conversion issues.

• Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? 

• Provide an updated schedule if changes have occurred. 

See attached.

Captone_Team_A2_-_LaSEEr11092019

Enes Palaz Status Update

• What did you personally accomplish this week on the project?

Tested and verified the code logic for making DAC code sleep until next points time is ready. I tested my code by generating square waves that were buried inside ILDA file format so observing aimed frequency verified that my calculations for timing between points were working. Found out the bugs in our ILDA structure with the help of Eliana and Jake. We suspect that format for ILDA files are in big endian and my initial assumption required testing with big endian changes made on my code.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

We are still on schedule for integrating software modules for next weeks demos.

• What deliverables do you hope to complete in the next week?

I am planning to be done with the DAC parts and successfully project an ILDA frame generated by our own software. Also planning to start some filtering on edge detection data so we can remove points from a frame if it contains more than 1200 points.

Jake Zimmer Status Update

• What did you personally accomplish this week on the project?
  • Constructed a rig to test the laser output from Enes’ DAC program with some test ILDA frames. I have included images of both the rig and the laser output below in the team portion.
  • Finalized and ordered the PCBs for the system.
  • Worked on the BOM for parts ordering.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • On schedule. I knew that getting the boards ordered would take some time and it is estimated to take less time than I expected.
  • As I said last week, I have also been working on writing some firmware for the STM32 microcontroller that we have decided to use. By writing this in parallel on a dev board, I have ensured that we should continue to be ahead of schedule for the near future.

• What deliverables do you hope to complete in the next week?
  • The boards should arrive at the end of next week and I will assemble and test them then.

Eliana Cohen Status Update

• What did you personally accomplish this week on the project?

• • Integrated Enes’ DAC/ILDA libraries, got them to compile with makefiles on raspberry pi, discovered new memory interface bug between software blocks.
  • Improved speed performance of Opencv frame reading by figuring out Camera resolution bug with Enes’ assistance [~50ms contour to ILDA frame conversion time, and ~30ms video to frame conversion time in multithreaded parallel computation, so our speed performance is now safely accomplished]
  • Tested ILDA output on laser hardware along with Jake and Enes

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

On track, though I need to finish up the memory copy bug, but should be able to get the ILDA to DAC pipeline completely finished by next week, which will allow us to test live frames on our laser hardware. We still need to work on figuring out some bugs/calibration with our physical hardware system, so that might take a priority.

• What deliverables do you hope to complete in the next week?

By next week, I should have completed:

1. Get a demo-ready implementation together for the hardware system
2. Fix the C++ Memory copy bug for the ILDA->DAC libraries.

Team Status Update

• What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

Laser being too thick, heat produced by the hardware are both concerns that degrade the quality of our image. Since laser thickness makes it harder to see a decent output, we need to be more careful with our laser selection for the future parts of our project. While doing initial testing, we also found our hardware produced a significant amount of heat (< 140 degrees), so we included a heavy-duty fan to reduce temperature. We recognize going forward we will need to install heatsinks, so we will begin part selection for heatsinks for this project.

• Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? 

Inclusion of heatsinks, potential fan, to reduce chances of damaging our hardware. This will increase the cost of our project slightly, but it isn’t a difficult or extremely costly change to make.

• Provide an updated schedule if changes have occurred. 

N/A

Enes Palaz Status Update

• What did you personally accomplish this week on the project?
  • Finished designing the interface for DAC module
  • Took a slack period for a midterm and homework for most of the work.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • I am on track with my gantt chart expectations

• What deliverables do you hope to complete in the next week?
  • Finishing the code for controlling DACs
  • Testing DAC code with the galvanometers

Jake Zimmer Status Update

• What did you personally accomplish this week on the project?
  • Designed the safety subsystem and implemented it in the schematic and PCB.
  • Redesigned the PCB to be a Raspberry Pi Hat.
  • Implemented changes to the DAC circuitry based on experimentation.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • I am on track although I expect that actually programming the safety subsystem will take longer than I allocated due to getting set up with a new toolchain.

• What deliverables do you hope to complete in the next week?
  • I will have a formal design review with Eliana Cohen.
  • We will order PCBs and parts.

Eliana Cohen Status Update

• What did you personally accomplish this week on the project?
  • Finished implementing the C++ implementation of the frame to ILDA converter, and benchmarked performance. Time to convert an image is now in the range of .1s – .08s, which is within specs for our conversion time. It might be nicer to get an extra factor of safety in regards to computation time.
  • Researched other improvements to make with the C++ implementation, and found further speedup could be accomplished via preallocating arrays rather than using vectors. 
  • Applied some minor code fixes to the C ILDA double buffer code.
  • Set up makefiles for compiling whole project and integrating the C++ and C buffer code together

• • Tried setting up the project on the raspberry pi hardware, but encountered some problems with configuring opencv. Currently investigating why certain environmental variables are having difficulty finding the built opencv libraries, but wasn’t able to figure this out by the end of the week.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

On track, though the problems I encountered with getting the OpenCV files to work on the raspberry Pi were unexpected. These aren’t really problems with the code, so I was still able to accomplish my deliverables for finishing up the libraries, and Enes was willing to give me some extra help fixing the environmental variables as I’ve been stuck on this for a couple days. It should be a simple fix, so it shouldn’t impair the schedule by too much.

• What deliverables do you hope to complete in the next week?

By next week, I should have completed:

1. Review the Safety Subsystem Hardware with Jake
2. Start working on and researching safety subsystem implementations. As I have exams next week, I’ll mainly focus on initial research and high-level design.

Team Status Update

• What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

No new risks.

• Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? 

The Frame to ILDA library was converted to C++ instead of Python, as discussed in the previous status update.

• Provide an updated schedule if changes have occurred. 

None

Jake Zimmer Status Update

• What did you personally accomplish this week on the project?
  • This week I did an in depth evaluation of the feasibility of the PID subsystem. PID was never a core goal of the project because the galvanometers come with their own PID controllers and so making our own was just an attempt to learn more about analog PID. The galvanometers use a pretty interesting method of providing feedback to the PID controller. The galvanometers use a light source, blocking element attached to the reverse side of the mirror output shaft, and two photosensitive elements to create a fast feedback system. After getting the device up and running with some basic test patterns and measuring the output I realized that there appears to be some non linearity in the system and that mitigating the effects of this would be incredibly time consuming. Therefore, my efforts in the future will be focused on other areas of the project.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • My schedule has drastically changed this week. I was slightly behind on the PID part of this system but now have changed focus to work on the required parts of this project.

• What deliverables do you hope to complete in the next week?
  • In the next week, I hope to have finished a raspberry pi sheild for the DAC component and finalized specification and detailed design requirements for the safety subsystem.

Enes Palaz Status Update

• What did you personally accomplish this week on the project?
  • This week, we received our DACs and breakout boards for DACs. I worked on soldering the DACs on their breakout boards which was challenging due to their size (3mmX3mm) and got help from Jake for creating protoboard steps which would allow me to create a reliable test bench on which I can test my code actively.
  • After all the soldering work was done, I worked on setting up a RaspberryPi 3 for testing DACs since Eliana has been actively working on performance improvements on our RaspberryPi 4 and we needed another Raspbian environment to develop in parallel.
  • Lastly, I coded initial test code which tried to read DEVICEID register from DAC and worked on debugging the setup with an oscilloscope and logic analyzer. I was able to successfully communicate with the DAC but hardware setup needs improvements to prevent noise on power and data communication pins.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • Even though, issues that I had with the DAC communication coding extended the completion time for DAC subsystem, we already had some extra time that was allowed for these situations so we will just move the deadline further and try to finish other parts in shorter time so we can still keep those safety days that we have.

• What deliverables do you hope to complete in the next week?
  • Next week, I am expecting to finish testing of the DAC and prepare interface for the actual code that will update DAC channels according to ILDA point structure. I am also expecting to start coding the part that reads from ILDA points and updates all 5 channels of the DAC and triggers those in synchronization.

Eliana Cohen Status Update

• What did you personally accomplish this week on the project?
  • Did research into speeding up the Python frame to points library. Initially I thought it’d be as simple as translating the contour object to point library into C, but then it turned out that writing python-wrapped C code is much trickier than anticipated due to the contours data structure not having any equivalent in C. 
    • After an initial attempt at the Python-wrapped C library, I investigated CPython’s ability to speedup C code. However, due to the use of numpy-arrays of variable size in the Contour data structure, I couldn’t achieve more than 4x speedup (~0.2 seconds to generate a frame) as I couldn’t define the numpy arrays at a fixed size, so CPython couldn’t optimize more of the code.
    • I then investigated Pytran for its integration with numpy arrays and its improved optimization as a result, but ran into similar trouble as the numpy arrays’ length weren’t decidable by compile-time.
    • After exhausting all of these options, I decided to rewrite these libraries entirely in C++ as it became evident that trying to work around the inefficiencies of Python was too time consuming. I’ve finished rewriting about half of the libraries, and should be able to produce a speed test of the C++ system by Sunday.

• • As we had our galvos and laser hardware, Jake and I tried to test out the system through a standard test-ILDA file and the frame-to-point library’s generated ILDA file. We purchased a parallel port to attempt to write the file to the board, however, the control board lacked documentation or drivers. After researching online and attempting to forcibly write the ILDA file to the board, we were unable to transfer any ILDA files to the test board. We concluded that we should wait for our DAC and DAC library so that we could bypass the black-box test board and instead write analog signals directly to the galvos. 

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Behind, as this Python speedup problem was much more complicated than I initially expected. However, this was a necessary problem to tackle, as the 10fps frame to point conversion is part of our requirements. I’ll have this finished by Monday, and since the DAC library isn’t finished yet, this library will still be ‘on time’ relative to when it needs to be done to test it with the other parts of this project. This also applies to the double-buffering library, but testing the double buffer library requires a complete Python-to-Points library.

• What deliverables do you hope to complete in the next week?

By next week, I should have completed:

1. Finish the C++ new version of the library and test it on the raspberry Pi. This will also involve setting up makefiles for easy testing.
2. Finish the double buffer library and integrate with the new C++ library

Team Status Update

• What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

Reducing the computation time to calculate contours in a video frame is required to meet our requirement of being able to display 10fps of live camera feed. Earlier in the week, we discovered that our python implementation was too slow, and sought other optimization programs to compensate for it. Trying to use python in the project was far too slow, and C was unable to handle some of the more complex data structures passed into the C interface by python, so we ultimately decided to rewrite our old code in C++. Most likely this will finally fix our framerate issue for this part of the project, we’ll need to research more thorough code optimization.

Since last week, we substantially reduced the risk of the PID feedback subsystem by doing an in depth analysis of the photocell-led section of the galvanometers. The feedback looks promising but there are still some minor concerns about getting this nonlinear system to perform adequately.

• Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? 

Since the PID controller is a significant risk to the project, we already have adequate PID controllers that came with the galvanometer, and PID control was never an explicit requirement nor core function of this project, the PID subsection is being made a stretch goal for this project. This means that we will focus on the DAC and safety hardware subsystems for the circuit portion of the project and leave the custom PID controller portion as a task if time allows.

We’ve begun to rewrite our old python libraries into C++ due to the aforementioned framerate issue. The main cost of this is the time required to do this, as rather than work on new libraries we’ve had to rewrite old ones. However, we’ve learned to avoid python and other higher-level languages for performance reasons, so for the rest of the project we should stick to C and C++ to avoid these sort of porting issues.

Since we switched to C++ for our whole pipeline, we decided to use go forward with the WiringPi library for SPI communication between RaspberryPi 4 and our DAC. This wasn’t exactly a design change but a key choice in our development.

To refine another requirement and highlight that both the frame-processing and frame-displaying should be at 10 frames per second, we’ve added this requirement:

• The laser shall draw frames at 10 frames per second because it is proximal to the classic animation framerate of 12 frames per second (which is proven to look visually fluid) while also being an easy divider of 30 frames per second, the modern framerate.
• The system shall process camera frames into DAC-readable points at 10 frames per second as well. Therefore, the hardware should not be stalled by the processing time of the frames-to-points library.

• Provide an updated schedule if changes have occurred. 

Due to unexpected difficulty in optimizing the python code and then eventual decision to switch to C++, the frames-to-point library schedule has been delayed.

The part of this project devoted to PID has now been changed to the DAC and Safety Subsystem.

Captone_Team_A2_-_LaSEEr-10-05-19

Jake Zimmer Status Update

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

Enes Palaz Status Update

Focus of my work was ordering DAC that we selected as a team and read through its datasheet in detail to make preparations for the interface that I am going to code for it in the upcoming week.

I read through the specifications on how to communicate with the DAC using SPI protocol and listed steps needed to successfully configure it. In addition to this, I noted the timing constraints for the DAC trigger so I can take those into account during the development phase.

I also worked on some research on SPI libraries that are available to use in multiple languages while we try to decide in choosing C or Python as our main language in the project based on performance constraints. As a result of the research, I decided to use WiringPi GPIO library because of its speed thanks to it being written in C. We want to utilize the performance advantage that comes with this.

Lastly, I made some research on other edge-detection algorithms that are based on deep-learning models that could improve our performance. I am mainly evaluating performance of HED(Hollistic Edge Detection) and Canny Edge Detection which is the method that we are currently using.

My progress is still on track since I am on hold for our DACs and breakout boards for these DACs to arrive. Therefore, I didn’t need to move anything in the schedule.

Hopefully, our parts are going to arrive on Monday (09/30/2019) so I can start reflowing those on breakout boards and start configuring. I am planning to be done with successfully outputting desired values on all 8 channels of the DAC trigger all of these in synch. The verification of this is going to be done by using a multi channel oscilloscope.

Eliana Cohen Status Update

• What did you personally accomplish this week on the project?
  • Integrated earlier OpenCV livestream output with ILDA conversion on our test raspi, and tested the time to convert a frame to ILDA.

This included making the initial test python code into modular libraries, and redefining the software stack structure for the project. An example of a frame converted in real time can be seen below:

• • Wrote pseudo-code for thread-safe double-buffer interface between video livestream and DAC interface.
    • Code is mostly finished, just requires a test-bench to benchmark its speed and thread safety. High-level structure of the double-buffer is below:

• • Assisted in some schematic review for PID design.

•  Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Mostly on schedule, after I integrated the video feed with the ILDA conversion python library, I found the python library was too slow (0.7 second frame conversion time) to keep up with our 10fps requirement. Hence, I will write up a C-version of this module by the end of this weekend, and so I will still finish this module-integration on time.

• What deliverables do you hope to complete in the next week?

By next week, I should have completed:

1. Ensuring live-generated ILDA frames work on our hardware system.
2. C-Version of the ILDA conversion library fully tested and completed.
3. Double-buffer library fully tested and completed.

Team Status Update

• What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? 

The PID controller is a big point of uncertainty for this project for two reasons. The first is that the PID controller is done in analog circuitry which means it is a bit difficult to make conceptual changes on the fly. Some laser systems seem to implement a PIDD controller with different D terms for high frequency and low frequency changes. To mitigate this risk, we will protoboard the system before getting manufactured boards. The second risk is that the feedback system for the PID controller is somewhat unusually constructed. The galvanometers use an LED and two photocells with a blocking element to derive a positional component. To mitigate this risk, we will scope the outputs of this element thoroughly to derive a good method of receiving feedback from it.

Computation time from video frame to DAC output is another concern. Preliminary testing revealed that trying to package the contour bytes from OpenCV to ILDA format in python were too slow (~0.7 seconds per frame), so we are switching to using C for this conversion for speed improvements. This should hopefully improve our speed limitations, but more testing is required.

In order to increase our performance more in case that we didn’t reach high enough frame rates in our C implementation, we added smoothing and filtering work in our schedule to develop a fast filtering step between contour detection and ILDA conversion to reduce the number of points converted. This method will mainly detect really small and unrelated contours that is not worth drawing so we can reduce the workload while not losing any visual context.

• Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? 

We changed our software stack structure after realizing Python was too slow 

            (for our 10/fps requirement) to handle contour to ILDA conversion. 

            The new software diagram is below:

We refined some of the requirements:

The laser shall draw frames at 10 frames per second because it is proximal to the classic animation framerate of 12 frames per second (which is proven to look visually fluid) while also being an easy divider of 30 frames per second, the modern framerate.

A frame shall consist of a number of points less than 12,000 (12kpps) divided by the framerate.

The laser shall achieve 90% accuracy on the 12kpps ILDA test pattern by visually conforming to 9 out of 10 of the features highlighted in the pattern specifications.

• Provide an updated schedule if changes have occurred. 

These are the changes to the current schedule: 09282019_sched