Is 4atm to big of a difference to get when I use the ideal gas law vs the van der waals equation?

This absolute amount of pressure does not play a significant difference.

At high temperatures and low pressures, its results deviate too little.

But at low temperatures and high pressures, it will change significantly and this difference will change from gas to gas.

In the image you can see the difference between some results from both equations.