This is not true , When you delete Data from the recycle bin the computer fills that space with Random Data = Free space

The data is still recoverable,

It is only not recoverable when : Other data is recorded on top of it , Or if the Drive is recorded with a bunch of 0's Also Known As a Format

I wouldn't worry too much about Shredding the Files unless the Data is very Sensitive E.G Bank numbers, Phone Numbers ... etc

You could use Ccleaner which cleans all Temp files from your computer and in the options you can Change the type of Deletion

Please see Piriform - Download CCleaner, Defraggler, Recuva, Speccy - Millions of users worldwide! For More Information

Hope This Helps,
Josh

In addition to what Shadowjk has said, may I add that eraser may be just the thing that you are looking for. I do feel that unless you have very sensitive private information a simple format will be all that is needed. You can even go one step further with a zero format as Shadow suggested.
Eraser

Eraser is suggested by most here. It is a very good product.