put a spring inside the adjusting bias screw, or replace the adjusting screw/knob combo, along with the center nut.
Or if you’re trying to keep it on the cheap side, then remove the bias screw and make the threads more rough and harder to turn, but still easy to adjust by hand, basically either make the nut tighter or the screw rougher.
If it’s that loose buy a new one. If it’s the unit I’m thinking of they are a poorly designed and dangerous piece of junk IMHO. If that little adjuster nut fails you loose all brakes. I wouldn’t use one. The one I designed is a simple block of aluminum that holds the brake rods and you adjust the front rod with a jam nut. You can’t set the bias on the fly but it won’t fail either.
Thank you both for the replies. I’m sans safety strap/cable between the brake pedal to the master cylinders as well, so it looks like I’ll be addressing the whole setup.