Cleanup LivyConf

While working on a separate thing I kinda started getting annoyed with LivyConf and how it replicates a lot of existing code from other places. We can merge some of that code and clean up its usage a big.