I'd go one step past what you have done so far (removing the rust lightly with an eraser).
I'd degrease it and paint the rusted/bare areas CAREFULLY with a neutrtalizing primer (auto parts store). This primer neutralizes the rust and it's a bit shinier than dull black. If you are careful it shouldn't look too bad. BTW, the bare spots add an original used look to the drum, and since it is pitted, you may just want to leave it oiled and call it a shooter.
Incidentally, I've had really good luck removing rust from various steel items (axeheads, tools, etc.) with oxalic acid (in crystalline form available at a pharmacy for a few bucks a pound) but never tried naval jelly because it's kinda pricey. The crystals can be mixed with warm water (4 heaping tablespoons or so to about a half-gallon) and will dissolve the rust in just a few minutes. You end up with clean bare metal after washing off a thin layer of yellowish slime. THAT BEING SAID, you gotta remember that bluing (as well as black Oxide finishes like DuLite) is a controlled rusting process, so there should be no doubt that anything that removes rust WILL affect the bluing also.
BTW, didja get a good deal on it??
john