Fluctuations in the brightness of the background radiation can lead to confusion with real point sources. Such background emission confusion will be important for infrared observations with relatively large beam sizes since the amount of fluctuation tends to increase with angular scale. In order to quantitively assess the effect of the background emission on the detection of point sources for current and future far-infrared observations by space-borne missions such as Spitzer, ASTRO-F, Herschel and SPICA, we have extended the Galactic emission map to higher angular resolution than the currently available data. Using this high resolution map, we estimate the sky confusion noise due to the emission from interstellar dust clouds or cirrus, based on fluctuation analysis and detailed photometry over realistically simulated images. We find that the confusion noise derived by simple fluctuation analysis agrees well with the result from realistic simulations. Although the sky confusion noise becomes dominant in long wavelength bands (> 100 μm) with 60 – 90cm aperture missions, it is expected to be two order of magnitude smaller for the next generation space missions with larger aperture sizes such as Herschel and SPICA.