## Fighting fireflies

### December 23, 2010

I’ve been playing around with path tracing on and off for longer than I care to admit. Although my dalliances never produced anything earth-shattering (and certainly nothing I’d be willing to post about on ompf), I’ve found it to be an endlessly fascinating subject. No matter how much I read about it, I only ever seem to be scratching the surface.

One of the biggest headaches I encountered were caused by “fireflies”: those bright pixels that can occur when a sampling a strong response combined with a small PDF somewhere along the path. For a long time, I was “fixing” these by hand painting over the offending pixels and pretending like nothing ever happened. Eventually though, the guilt of this gnawed away at me long enough to motivate finding some kind of better solution.

My first thought was to write a filter that estimated variance in an image and replace any “bad” pixels it found with a weighted average of their neighbours. Luckily, my second thought was of shadow maps, the only other context in which I’d read about variance before. Based on the ideas in that paper, I accumulate two separate per-pixel buffers: one storing the running sum of the samples and the other storing the sum of their squares. Having these two buffers then makes it trivial to compute the sample variance of each pixel in the image.

My path tracer already had support for progressive refinement, so it was straightforward to add a separate “variance reduction” pass that would run at the touch of a button. During this pass, the N pixels with the highest variance are identified and oversampled a few hundred times, which hopefully reduces their variance sufficiently. If not, I just run the pass again.

As an example, here’s a render of the Manifold mesh from Torolf Sauermann’s awesome model repository, stopped after only a few paths have been traced per pixel:

I’ve highlighted a few areas that contain fireflies and below is a comparison of those areas before and after running the variance reduction pass:

And here’s the complete result of running the pass; it’s still noisy, but the pixels with particularly high variance have been cleaned up reasonably well:

I’m not too hot at statistics, but I would guess that this adds bias to the final result, which is frowned upon in some circles (but not others). Admittedly, a better solution would be to simply not generate so much variance in the first place, but this will do as a kludge until then. At least it’s better than painting pixels by hand!

Ok, since this post was mostly just an excuse to dump some results from my path tracer, here they are.

The XYZ RGB Dragon from Stanford’s 3D Scanning Repository:

A heat map of the BIH built for the same model.

The other Dragon from Stanford:

Manifold again, from jotero.com.

Some Stanford Bunnies:

Crytek’s updated version of Marko Dabrovic’s Sponza model:

And Stanford’s Lucy, just to prove that I can:
I’ve not been able to find any close up renders of Lucy to compare with, but I believe that the “pimples” on the model are noise in the original dataset.

### 4 Responses to “Fighting fireflies”

1. Krys Says:

Interesting, I like the way you fight against firefly. It is a very simple but efficient way… maybe …

But don’t you think that you will remove some caustics… or simply decrease the convergence speed ?

Another small question, which material do you use for the Manifold model ? which BSDF and which parameters… I will be interested to compare your rendering with mine ðŸ™‚ If you allow me ðŸ˜›

The technique I describe only affects pixels with high variance (i.e. those that have been poorly sampled).

Sampling those pixels more times will only improve the accuracy of the result, so true caustics will not be removed. Since the number of pixels resampled is tiny compared the the image size (around 0.01% for a 720p image), convergence speed is not really affected.

I don’t know if I still have the scene description for the manifold scene; I’m kinda busy this week, but I’ll see if I can dig it out over the weekend.

I got the manifold model from the forum on jorato.com (look for “manifold.rar” on that page).

3. pl01 Says:

Thanks Tom,

Very interesting… for sure, I will give a try ðŸ™‚

4. squeen Says:

Another hack that make a big difference with fireflies as well as anti-aliasing of very bright light source near the edges of objects is to super-sample (i.e. draw a larger version of the image), and then down-sample (shrink) the image in LDR instead of HDR. The fireflies are then averaged at the clamp value.

I like your variance reduction idea. I was planning to do something almost identical myself, but had not gotten around to it. I was going to keep a running sample-variance buffer for each pixel and progressive shoot more rays at the high ones until the 1-sigma value dropped between a LDR color threshold.