get the clean out screw in things, i forget what they are actually called, you basically make it a screw top, its still air/water tight too, maybe put a pair of good size cheap pliers next to it.
but first, get a 6 inch pvc, dig the hole big enough for that, put the 6" into the hole, then insert the 4 inch inside the 6, otherwise you have to dig dang near the wholle thing back up to get it out in 10 years, once everything settles it doesnt just slide out of the ground by grabbing the top. increase the outer pvc for whatever pvc you use for the storage part.
I cant see PVC ever exceeding the cost of a rifle size pelican case. not to mention that makes it wider than a pvc. unless you buy it pre cut or something, buy it in 10 or 20 foot lengths and its fairly cheap, then cut to what you want
cheaper than dirt does have some burial grade ziplock type bags btw, but they are like 80 bucks for rifle size as I recall.