A Low-Cost Integrated Perception and Manipulation Framework for Indoor Target Localization and Grasping

Abstract

Robotics and Artificial intelligence are becoming substantial tools in shaping the future of agriculture and addressing its challenges. Empowering small-scale agriculture, our paper tackles the inefficiency and wastage in fruit harvesting for small-scale growers. Our proposal integrates a robotic arm with machine learning models that detect and classify fruit ripeness using colored images. A camera captures fruit data, and the deep learning models determine the fruit type and its ripeness. The robotic arm picks and sorts fruits into designated areas for ripe and rotten fruits. The results of our proposed work demonstrate a successful picking time of 30.46 seconds per fruit. By leveraging multithreading, the system achieved significant improvements, resulting in a frame response time that is 12 times faster and enhancing overall operational efficiency. The classification model attained a test accuracy of 96.27% and demonstrated robust performance metrics. The key novelty of HarvestMate lies in its end-to-end autonomous harvesting cycle on a single low-cost embedded platform. An efficient, platform-specific optimization using a precomputed reachability matrix reduces target filtering time by approximately 96%, and a two-stage ripeness verification pipeline, addressing an accessibility gap for small-scale indoor growers currently unserved by commercial solutions.