We could do auto limit 5000 at the result fetch and/or even SQL (optional). Then we could have warnings when we know we have more and the user wants to see/download, and he could extend the limit.
- Caching service interface: e.g. https://docs.djangoproject.com/en/2.2/topics/cache/#filesystem-caching
- Format: Parquet via Panda?