Caps would be a problem, how would you improvise ignition?
If we go a little further in the way back machine, we find flintlock firearms, even further gives you wheellocks. Wheellocks would be a little more reliable, but more fragile and complicated, and slower to load.
What the OP is really looking for is, a stainless steel flintlock blunderbuss with a synthetic stock. All ignition and propellant components could be improvised with natural materials, and no bullet molds or specific materials neccessary, the world is your ammo pouch.
If it fits, it shoots.