We present a new method which allows a swarm of robots to sort arbitrarily arranged objects into homogeneous clusters. In the ideal case, a distributed robotic sorting method should establish a single homogeneous cluster for each object type. This can be achieved with existing methods, but the rate of convergence is thought to be too slow for industrial application. Previous research on distributed robotic sorting is typified by randomized movement with a pick-up/deposit behaviour that is a probabilistic function of local object density. We investigate whether the ability of each robot to localize and return to remembered places can improve distributed sorting performance. In our method, each robot maintains a cache point for each object type. Upon collecting an object, it returns to add this object to the cluster surrounding the cache point. Similar to previous biologically-inspired work on distributed sorting, no explicit communication between robots is implemented. However, the robots can come to a consensus on the best cache for each object type by observing clusters and comparing their sizes with their remembered cache sizes. We refer to this method as cache consensus. Our results indicate that incorporating this localization capability enables a significant improvement in the rate of convergence. Various operational parameters are assessed in experiments using a realistic simulation of our targeted robotic platform. A subset of these experiments is also validated on physical robots.