muzzle brake build up?

[DirtRacer151]

I was shooting my ceiner kit in my ar for the first time this weekend and noticed my muzzle brake had about 1/16 worth of build up stuck on the ports after about 200 .22lr rounds. What the heck causes this and what is it? It was near impossible to scrape off but 20 rnds of hot loaded .223 did the trick in shaking it loose little by little every shot. Im affraid that if left on there after a long 500+ rnd range session it might be near impossible to remove. Any thoughts?