I don't know exactly how the QGIS interpolation plugin works. Someone else may be able to give more specifics on the tool itself, but with a little testing and observation I think I know what is causing the result you're seeing. Based on the screenshot of the dialog box I see at the documentation, there's no user input parameter for a search radius with the interpolation tool.
I downloaded your data and tried running it through the ArcGIS IDW tool to see if I got similar results, and I did. I note that I have some additional/different input options available - specifically I can use Power while you have Coefficient, and I have a the ability to put in a search radius. More on Power vs Coeffcient can be found at this other GIS.SE question: How to pick the coefficient in Inverse Distance Weighted method? And I believe this is partially related to what you're seeing.
Then there's the search radius. My default setting is a variable radius with a point count and an option for a maximum distance. If I don't specify one, that means for every output raster cell it's going to go find the closest n (point count) points to get the cell value. My default was 12. To help see what was going on, I symbolized the points on a graduated color scale by value.
Zooming in to the area of your 1/2 example, you can see there are some higher values on the data points at those edge curves. If QGIS interpolation is working the same way, some of those cells in the upper right are pulling their values from the n nearest points, which are going to be the points on those curves, causing their values to be higher. The close you get to the curve, however, the more zero value points available to interpolate from, which drags the cell value down. In this case having no data points out there is actually increasing the values, because there are no zeros to counteract the higher values that are in range.
A similar thing occurs are your 3/4 example. Symbolizing your points by value range classification might help you see what's going on. It's a good idea try a couple of different values for things like search radius or coefficient when interpolating and examine the resulting surface. In your case (according to the question linked above), you might try a smaller coefficient, but I don't know that will completely eliminate the effect and it's a bit of a cartographic balancing act.
Also, I note you've used the tag heatmap. There is a heatmap plugin for QGIS (which allows for a search radius and a weight by field), but it's not the same as interpolation. The heatmap plugin looks at a density of points, not interpolating the values of points. You may already understand the distinction, but for completeness and future viewers, I wanted to make sure and point this out. See this related question: How to build effective heat-maps?
