When shifting without the clutch you are basically matching the engine’s speed with the transmission’s speed. When you upshift, the transmission will be going slower than the engine; therefore, you have to let the engine slow down before you can successfully put it into gear.
When you downshift, your engine will be rotating slower than the transmission; therefore, you must blip (give a bit of gas) the throttle to increase the speed of the engine to match the speed of the transmission.