Mar 19th:
New, fully realized beautiful setup. Thank god for heavy weight bearing adhesive velcro straps and sticky putty.
Pixel diff is working, at least for all the situations I used it in excluding one (crushed tomatoes) (may still need fixed light if we’re doing it in different lighting conditions). No major issues to report there.
Tested two situations, one where the item was directly under the camera, and one where it was off to the side with anti-distortion of the image.
Overall results were fine, but definitely worse than the photos taken with the I-phone camera. also, the side shot was definitely worse then the center shots for most items Here is the confusion matrix based on the number of matches we saw for each situation. (the number shown is the number of (matches)/(number of matches for the best match for that particular query image))
Sideshot:
| Applesauce | Milk | CrushedTomatos | Shredded cheese | Spaghetti | BakingPowder | Yogurt | Beans | Cereal | Crackers | |
| Applesauce | 0 | 0 | 0.40678 | 0 | 0.147959 | 0.820513 | 0.869565 | 0.083333 | 0.7 | 0 |
| Milk | 0.462236 | 0.68254 | 0.20339 | 0.093023 | 0 | 0.846154 | 0.942029 | 0.375 | 0.766667 | 0.113861 |
| CrushedTomatos | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| Shredded cheese | 0.954683 | 0.936508 | 0.559322 | 0.937984 | 0.908163 | 0.974359 | 0.956522 | 0.166667 | 0.772222 | 0.935644 |
| Spaghetti | 0.960725 | 0.952381 | 0.254237 | 0.984496 | 0.80102 | 0.974359 | 0.913043 | 0.458333 | 0.844444 | 0.970297 |
| BakingPowder | 0.845921 | 0.809524 | 0.694915 | 0.728682 | 0.739796 | 0 | 0.884058 | 0.125 | 0.811111 | 0.79703 |
| Yogurt | 0.867069 | 0.857143 | 0 | 0.806202 | 0.816327 | 0.871795 | 0 | 0 | 0.75 | 0.866337 |
| Beans | 0.975831 | 0.968254 | 0.644068 | 0.984496 | 0.97449 | 0.948718 | 0.971014 | 0.125 | 0.811111 | 0.980198 |
| Cereal | 0.975831 | 0.968254 | 0.644068 | 0.968992 | 0.969388 | 0.871795 | 0.898551 | 0.208333 | 0 | 0.985149 |
| Crackers | 0.97281 | 0.936508 | 0.237288 | 0.945736 | 0.954082 | 0.948718 | 0.913043 | 0.208333 | 0.783333 | 0.712871 |
Center Shot:
| Applesauce | Milk | CrushedTomatos | Shredded cheese | Spaghetti | BakingPowder | Yogurt | Beans | Cereal | Crackers | |
| Applesauce | 0.151487 | 0.139073 | 0.0261 | 0.028105 | 0.082428 | 0.001076 | 0.002422 | 0.004927 | 0.041316 | 0.295754 |
| Milk | 0.081465 | 0.04415 | 0.035048 | 0.02549 | 0.096742 | 0.000923 | 0.001076 | 0.003359 | 0.032135 | 0.262079 |
| CrushedTomatos | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Shredded cheese | 0.006865 | 0.00883 | 0.019389 | 0.001743 | 0.008885 | 0.000154 | 0.000807 | 0.004479 | 0.03137 | 0.019034 |
| Spaghetti | 0.00595 | 0.006623 | 0.032811 | 0.000436 | 0.01925 | 0.000154 | 0.001615 | 0.002912 | 0.021423 | 0.008785 |
| BakingPowder | 0.023341 | 0.02649 | 0.013423 | 0.007625 | 0.025173 | 0.005996 | 0.002153 | 0.004703 | 0.026014 | 0.060029 |
| Yogurt | 0.020137 | 0.019868 | 0.043997 | 0.005447 | 0.017769 | 0.000769 | 0.018568 | 0.005375 | 0.03443 | 0.039531 |
| Beans | 0.003661 | 0.004415 | 0.01566 | 0.000436 | 0.002468 | 0.000308 | 0.000538 | 0.004703 | 0.026014 | 0.005857 |
| Cereal | 0.003661 | 0.004415 | 0.01566 | 0.000871 | 0.002962 | 0.000769 | 0.001884 | 0.004255 | 0.13772 | 0.004392 |
| Crackers | 0.004119 | 0.00883 | 0.033557 | 0.001525 | 0.004442 | 0.000308 | 0.001615 | 0.004255 | 0.029839 | 0.084919 |
A few standout items that were confused with each other: Applesauce just generally had a larger amount of descriptors, so there were more matches. I think there was also an issue with crushed tomatoes where the pixel diff failed. Overall, I don’t think there’s anything that can’t be handled on a case by case basis. However, I’m somewhat concerned that our ability to recognize when we have an unsupported object will be more difficult.
Overall, I’m mostly happy with the state of the CV component, it’s at least minimally functional. I will likely move to help working on the other stuff to get the demo working. I may also try adding some checks for the stuff that was confused, if I can find something simple.