2.94 KB
Newer Older
David Daish's avatar
David Daish committed
1 2 3 4 5 6
How does this repo work? Not even I know.

# Todo

- [x] Create a plank like block of relevant proportions.
- [x] Create several of them, with random poses.
David Daish's avatar
David Daish committed
- [x] Have a depth camera and logic camera look at the objects, and produce data.
- [x] Create distracting objects.
David Daish's avatar
David Daish committed
- [x] Have the camera move around randomly.
- [x] Extract data automatically.
David Daish's avatar
David Daish committed
- [x] Script and automate this process, to generate a dataset.
David Daish's avatar
David Daish committed
12 13 14 15 16 17 18 19 20
- [ ] Write neural network for estimating pose information
- [ ] Get estimate accuracy > 95%
- [ ] Be able to estimate poses of multiple objects
- [ ] Be able to estimate the size and pose of arbitrary planks

ideas for increasing accuracy:

- [ ] Change input from depth data, to point clouds
- [ ] Add RGB data to input
David Daish's avatar
David Daish committed


# Issues

25 26 27
- ~~If the simulation does not last long enough, it will end before the files can
  be written to disk.~~ Resolved by using `.close()` method on opened files.
- ~~I suspect the main bottle neck is the startup and shutdown of the gazebo
  program, and that if the program isn't closed each iteration of the
29 30 31 32 33 34 35 36 37 38
  simulation, it can go much faster.~~ Assumption was correct. By running the
  simulation multiple times without destroying the docker container, the sim
  can be generated much faster.
- Apparent slowdown over time.
    - ~~Cannot see any balooning models or other entities with World.Models(),
      WorldState.SDF().ToString(), or PhysicsEngine.DebugPrint()~~
    - ~~Possibly a balooning number of cam_record plugins?~~ Nope, I removed the
      model entirely, and it still didn't change.
    - Screw it, I give up. Resolved by ending and restarting the program before
      the slowdown gets too bad.
39 40 41 42 43 44 45 46 47 48 49
- Sometimes, multiple planks will appear in rendered images.
    - It doesn't happen every batch.
    - The extra models are not listed with a call to `WorldPtr->Models()`.
    - When the issue appears once, it'll appear across all the other images.
      Alternatively, it might just appear in most of the other images. It's
      hard to tell.
    - The extra models can be seen in the depth images.
    - The scene does not appear to change, when accessed by `WorldPtr->SceneMsg()`.
    - The ghost models can be accessed using `ScenePtr->WorldVisual()->GetChild()`.
    - I cannot find the cause, so I have resolved the issue by creating a simple,
      hacky system that ends the process if more visuals are detected than expected.
David Daish's avatar
David Daish committed
50 51
    - For a reason I cannot identify, the hacky solution will produce blank white 
      images as the only produced dataset.
52 53 54 55
- The NN would have no way to diferentiate the pose differences of a plank that
  is symmetrical. The serialisation process needs to limit it to just a few
  directions, such that the NN doesn't need to determine the exact pose, just the
  relevant ones.
David Daish's avatar
David Daish committed
56 57 58 59
- The strange blank datasets need to be removed from the data before it trains the
  neural net.
- The only way to prematurely end the serial batch dataset generation is to close
  the terminal window, which isn't very graceful.