In the end our project works fairly well but did not quite meet the original criteria of being able to pick up and orient the car from any position. The program in the end only picks up the car and places it on the board, the pieces themselves must be placed on the table in the orientation that they will be placed on the board.
Our main difficulties as mentioned on other pages was with creating tags that could be accurately read by the camera. To this end the size of the pieces had to be modified to accommodate the larger ar tags. Even when we upgraded to larger ar tags the camera on Baxter would read the ar tag location wrong and display it off to the side of the real position in rviz. We fixed this by closing and re-opening the camera and restarting Baxter.
One hack of our program is the hard coding of the scanning positions above the car and above the board. It would have been nice to be able to have Baxter calibrate these appropriate spots so that we were not dependent on table height and positioning. We also place the pieces in their ultimate orientation instead of having Baxter rotate them.
If we had more time we would have liked to work on getting the orientation to work properly. We would also have liked to have Baxter solve the game after he set it up, in case the player can not figure out the solution. Additionally, we would like Baxter to be able to clear the board after the user is done playing.
