This is because they are not allowed to. Doctors are being used more as technicians than as critical thinkers and experts. The decisions are made by the insurance companies. I think this is a travesty and hurts a lot of patients.
The trend now is towards "evidence-based medicine" which means that we only want to use things that have been shown to work. Of course no one wants to spend time or money on a treatment that has no effect, but this approach has some serious drawbacks. If a treatment works for 51% of people, it will be then used for 100% of cases. Never mind that your doctor has seen this a thousand times before and knows that treatment B would probably work better for you. If treatment A works better for more people, then it must be used on all. This system reduces people to statistics and neglects individual differences.
And it is not always cost effective. I tore cartilage in my knee several years ago. My orthopedist told me, "We'll send you for an x-ray, and when that doesn't show anything, we'll send you for an MRI which will show me the location and extent of the tear." I asked him, "If you know the x-ray will be useless, why don't we skip straight to the MRI?" To which he answered, "that's the only way the insurance will pay for it". How many unnecessary tests are done in the hopes of avoiding other, more-expensive tests? How many patients wind up requiring more care in the end because the insurance company was trying to save money at the outset.
Yes, we want to reduce waste and inefficiency, but I don't think the evidence-based model is the best one to do this. Our doctors should be allowed to exercise their professional judgment and do what is best for each individual patient. We need to demand that insurance bureaucrats stop being in the position of making our medical decisions and put that responsibility back where it belongs, between the patient and his/her doctor.