Film Simulation Software – Survey Research Results

LongShadow

Introduction

Now, that I am done with my bachelor thesis, I have finally found some time to write about the results of the study and about the results of the survey I conducted (You can still take part of it by following this Link).

I am into photography since over 6 years and I always found the look from old photographs interesting and unique. I came to the topic during a conversation with my professor, as we were talking about the unique style from polaroid photos. The topic was also influenced by the black & white photographs from Sebastião Salgado, one of my biggest role models (and… yes… I am in my creative black & white phase…). So, I choose the film Kodak Tri-X 400 for my research. Everything turned even more interesting as I found an article in which Salgado describes, how he “renounced” to the Kodak Tri-X 400 since 2008, by simulating the style of this film on the PC using DxO FilmPack.

During my research, I had to deal with a wide spectrum of topics: I learned a lot about old photography methods, about the similarities of digital and film photography, about the factors which affect the look of the photograph, about how film simulation software works, and about the creation and analysis of an online survey as well. The survey started some discussions in Germany, Argentina, Finland and China, and some of you also contacted me. You showed me your interest about the topic, asked me about the results (That’s why I am writing this post), and you gave me some constructive criticism about the form of the survey. Your feedback is for me one of the most beautiful experiences I had during this whole study!

So, enough preface, let’s dive into the survey. First I will answer the following question:

Why a survey?

As you may know, our perception is subjective, and in photography there is no “true” or “false”. Since I wanted to find out how good DxO FilmPack simulates film, I decided to start a survey where the participants should decide which photographs were made using real Kodak Tri-X 400 film. This would lead to more opinions about the photos comparison, and with the results I would be able to analyse if knowledge and experience in photography has an influence while comparing both types of photographs.

Gear used

For the photo comparison, I created a total of 24 photos (12 with a film camera, 12 with a digital camera). The Kodak Tri-X 400 photographs were made using a Canon AE-1, with a 50mm f/1.8 lens. The digital photographs were made using a Nikon D7000, with a Sigma 18-35mm f/1.8. Since the Nikon D7000 has a crop factor of 1.5, I was able to shoot at 35mm for maintaining the view angle of the 50mm lens on the Canon AE-1.

I shoot all photographs using the exactly same settings on both cameras (focal length, aperture, ISO). That’s why you should be able to see a difference on the depth of field… Next time I should consider the difference on the depths of field while using different sensor/film sizes… ;)

Survey criteria

As I already wrote, I wanted to investigate if people with knowledge and experience in photography have a “trained eye” for differencing the photographs made using Kodak Tri-X 400 from the digital film simulations. So, I started by dividing the main group (all persons) into smaller groups:

  • People with photography knowledge
  • People without photography knowledge

I also made the assumption, that all people who take photos do have photography knowledge. Even if you are just taking photos with Instagram. Otherwise (if you don’t take photos at all) you don’t have photography knowledge.

After that, I continued by dividing the group of people with photography knowledge in two more groups, based on their income (Defined by PhotoShelter):

  • Professional Photographer (If you earn more than 50% of your income from photography)
  • Enthusiast Photographer (If you earn less than 50% of your income from photography)

This should answer, if the photographer type has an influence on their selection.

To find out, if knowledge also plays an important role, I decided to ask the following questions:

  • Have you ever worked with film?
  • Do you use software for film simulation?
  • Do you use vintage apps on your smartphone for simulating old film style?

Of course, there are a lot more of criteria for evaluating the results, like the monitor resolution, the country where the participants live, which camera they usually use, etc. But for this study, I decided to just stop right there and evaluate the results based on the criteria I described.

Survey conditions

I created an online survey, using Google Forms and saving the results to a Google Spreadsheet, so I can check it anytime. Creating an online survey has a lot of pros: Everybody (with an internet connection) can take part of it, it’s anonymous, it’s easy to share, and so on… But nowadays we have different types of monitors, resolutions, DPIs, etc., so it’s very hard to control the viewing conditions of everybody’s display… I decided to export the photos in maximum of Full HD size (1920x1080px), using JPEG format and 100% quality (so the grain is still visible). It’s also important to mention, that this whole survey and study is about the overall impression of the photographs, and not about a deep analysis of the grain structure using a magnifying glass! That would be a topic for another research, where photographs could/should be printed (of course, the survey form would then be different too).

So: The results of this survey are just valid with the conditions I described and should not be generalised for all film types and film simulation programs!

Survey results

At the moment when I evaluated the results of the survey, it had a total of 114 participants. Right now this number has increased, since more people took part of it, but I am writing about the results with the 114 participants. If there are more still people taking part of it, I will write in the future about the new results ;)

For the evaluation of the results, I just counted the right answers. That means, I just counted the participant’s selections, where they chose the photographs made with film.

First of all: the general results. The average of right answers of all participants is 6,24 (of 12). This means that the participants could identify the film photograph just in 52% of the cases. So it can be deduced, that people can not really difference between a film photograph and a digital film simulation, and that DxO FilmPack 5 simulates the style of the Kodak Tri-X 400 good enough to fool/cheat the human eye.

As I wanted to find out more about their selections, I started to dive deeper in the results and I continued by dividing this main group as I described earlier. The following pie chart shows the division of the participants in persons with and without photography knowledge:

Participant Groups

As you can see, most of the people who took part of the survey have photography knowledge (A total of 104 participants), and just 10 persons don’t really take photos. And that’s pretty ok, since most of us take photographs almost everyday, with a SLR, compact camera, or smartphone.

I continued by dividing the group of persons who take photos (from the 91.2% above) in two groups based on their level, as I described earlier: photo enthusiasts, and photo professionals:

teilnehmerniveau

A total of 95 participants consider themselves as “photo enthusiasts” and just a few (9) as professionals. Since the difference between these groups is so big, I was not able to give the professional photographers so much relevance during the evaluation. For that, I would need more professional photographers taking part of this survey. If the difference between two groups is smaller or if both groups have the same number of participants, then the results would be more representative.

 

After that, I defined a total of 9 different groups, according to their answers (thanks boolean algebra, I can finally use you):

 

Group Have you ever worked with film? Do you use film simulation software? Do you use vintage apps on your smartphone for simulating old film style?
A No No No
B No No Yes
C No Yes No
D No Yes Yes
E Yes No No
F Yes No Yes
G Yes Yes No
H Yes Yes Yes
I (persons without photography knowledge)

 

I had three expectations on the analysis of the results:

  1. The average of correct answers from Group I (Persons without photography knowledge) would be the lowest.
  2. The average of correct answers from people who worked with film (Groups E, F, G, H) would be much greater than the average of correct answers from people who haven’t worked with film (Groups A, B, C, D).
  3. The average of correct answers from Group H would be the highest, and the average of correct answers from Group A would be the lowest.

The next bar chart shows the average of correct answers from the groups described above:

GroupResults

The first thing you will see on the chart, is the peak on groups C and D, with an average of 8 right answers in each! Then, you may deduce, that people belonging to this groups can differentiate better between both types of photographs, right? So, if I just use film simulation software on the PC (Group C) for putting some filters on the pictures I take, I have a better eye to see the differences, right? Or if I also use Instagram (Group D), then I have a trained eye, right?

 

Wrong. But why?

 

Because there are only 2 persons in each of this groups, so their answers are not really representative. For clarifying the results plotted on the chart, take a look at the next table. It shows the average of correct answers and the number of participants of each group.

 

Group Nr. of participants Average of correct answers General Average
A

17

6 6.54
B 12 7.08
C 2 8
D 2 8
E 27 6.44 6.33
F 20 6.05
G 9 6.22
H 15 6.60
I 10 4.20

4.20

 

Analysing the results, and comparing them with my expectations, there are a few things I can say about that:

  • Expectation Nr. 1: Right! Group I (persons without photography knowledge) shows the lowest average of correct answers. This might be, because they just made some random choices about the photographs (maybe they don’t know where to look, what to compare, etc).
  • Expectation Nr. 2: False! The general average from group A, B, C, D is even higher than the general average from group E, F, G, H! The first group has an average of 6.54 and the second one of 6.33. This is actually a very small difference (3.2%), and the first group (33) had less participants that the second one (71). Maybe, if the first group have had more participants (same number than the second group), the general average may have decreased… So, this means, that experience with film photography doesn’t really leads to better results on the comparison.
  • Expectation Nr. 3: Right! Group H’s average is higher than group A’s, and in this case it is valid, because both groups have almost the same number of participants (17 against 15). This means, that people who have already worked with film, use film simulation software on PC and use vintage apps like Instagram, can differentiate better than people who don’t.

 

Conclusion

On the one hand, I made following findings by analyzing the results of the survey research :

  • People are able to see the differences between a real film photograph and a digital film simulation in the viewing conditions defined for this research, in just 54,35% of the cases.
  • People who have photography knowledge, can differentiate between both types of photographs in 53.50% of the cases.
  • People, who don’t have photography knowledge, can just differentiate between both types of photographs in only 35% of the cases.
  • Experience in film photography does not really lead to a better or trained eye for differentiating both types of photographs.

On the other hand, it has to be considered, that the differences between the averages of correct answers from the groups are not so big. There could exist two reasons for this:

  • DxO FilmPack 5 simulates the style of Kodak Tri-X 400 so good, that we can just barely see a difference between the photographs, or
  • the chosen method for this survey and the viewing conditions (display, resolution, ambient brightness, photo quality, photo size, …) are not adequate, so that it leads to a wrong interpretation of the results.

Once again, The results of this survey are just valid with the conditions I described and should not be generalised for all film types and film simulation programs!

 

Some takeaway points

The goal of this research was to answer this question:

Can the human being recognize the difference between a real film photograph and a digital film simulation under the given viewing conditions?

For answering this question, I made a research about the foundations of film and digital photograph, and about which factors can influence the style of the results. I found that both the properties of a film as well as many external factors in the photographic process have an impact on the result, and that there is no absolute/unique “film style”.

In this work, I just examined the style of one film, and used just one film simulation program. Future work should take into account a larger number of films, which can be simulated with different film simulation programs. This would serve to check the accuracy of film simulations. If they are not good enough, suggestions for improvements to the software manufacturers can be sent, so that programs can be optimized.

In addition, the survey research can be used as a basis for future studies of this topic. They could be improved by taking into account further demographic characteristics of the participants. And the more participants, the more precise the results will be.

If further investigations confirm the question asked in this study, then this would mean that film simulation programs have met in this context one of the main intentions of computer science:

Confirm further investigations the question asked in this study research question, then this would mean that film simulation programs have met in this context one of the main intentions of computer science: the simulation of the reality.

 

This whole study also encouraged me to think about the philosophical aspects of film photography. I share the opinion of other photographers (like Eric Kim) about this: Sometimes we don’t really use film because of the good quality or because of the unique style it achieves, but much more because of the moments we experience by shooting with film. We think a lot more about composition and about the light situation (among other aspects) before we press the shutter. It slows us down and we perceive our “reality” even differently. And this experience is something that can’t be simulated (Or at least not yet).


Thank you very much for reading! If you found this article interesting or useful, please leave me a comment or write me a mail :) Also feel free to share this article! I would be very interested in a deeper discussion about this topic!

You can still take part of the survey by following this link.

Link to the previous post about this survey research.

You can also find me on facebook.