Welcome to Calguns. You'd probably have a lot of fun starting off with a .22 pistol or rifle. Ruger MKIII pistols are a great choice for beginners, as well as the Ruger 10/22 rifle. Cheap, fun, accurate, and ammo is dirt cheap compared to centerfire ammo.
Familiarizing your child with firearms at an early age will vastly reduce the chance of accidental discharge at home. Until a child actually shoots a gun, their only concept of guns is what they see on television. In a child's eyes the Hollywood version of guns looks like glamorous "power givers" that good guys have fun with while making bad guys go away. Once a child actually shoots a gun it puts it in a perspective that they can easily understand that guns are not toys and lets them grasp the concept of how devastatingly powerful a firearm can be. As long as they're under strict supervision
, it's a good way to instill the concept of acting in a responsible, clear-minded, focused way, while following directions from the parental figures.
Also, most of the young kids I see at the shooting ranges with their parents are having a great time.