Egospheric SpatialMemory encodes the memory in an ego-sphere around the agent, enabling expressive 3D representations . ESM can be trained end-to-end via imitation or reinforcement learning, and improves both trainingefficiency and final performance against other memory baselines . The explicit egocentric geometry also enables us to seamlessly combine the learned controller with other non-learnedmodalities, such as local obstacle avoidance. We further show applications tosemantic segmentation on the ScanNet dataset, where ESM naturally combinesimage-level and map-level inference modalities. ESM provides a general computation graph for embodiedspatial reasoning, and the module forms a bridge between real-time mappingsystems and differentiable memory architectures. We show that ESM has applications to semantic segmentation

Author(s) : Daniel Lenton, Stephen James, Ronald Clark, Andrew J. Davison

Links : PDF - Abstract

Code :


Keywords : esm - memory - egospheric - show - applications -

Leave a Reply

Your email address will not be published. Required fields are marked *