Most of OpenStack services use an application to offer persistence and store ephemeral datas (like tokens). Memcached is one of them and can scale-out easily without specific trick.
To install and configure it, you can read the official documentation.
Memory caching is managed by Oslo-incubator for so the way to use multiple memcached servers is the same for all projects.
Example with two hosts:
memcached_servers = controller1:11211,controller2:11211
By default, controller1 will handle the caching service but if the host goes down, controller2 will do the job. More informations about memcached installation are in the OpenStack Compute Manual.
