We have a guest, Jas “Fizzi” Laferriere (Twitter @Fizzi36, Email: Fizzi36@gmail.com), who put in lots of interesting research into input lag.
EDIT: I have written a short follow-up to this article with recommendations about monitors and devices check it out here: Reddit Post
Table of Contents
2. Console “Lag”
3. Synchronization by Sound
4. The Test Method
6. Result Analysis
7. Screen Quality Perception
CRTs oh how we love thee…
Yes we all know that CRTs have the lowest input lag of any display ever made. The problem with this longstanding fact is that any non-CRT screen is shunned by smashers as if they might contract the bubonic plague instantly upon use.
There has been talk of LCD monitors being used at upcoming international events such as Evo. This talk, as expected, has elicited a negative reaction from many smashers. The primary goal of this article is to outline just how close these monitors can get to the response time of a CRT, what the effect of lag is on gameplay, and to dispel some myths people have about LCD monitors.
In order to determine if a particular monitor lags, people will often try the subjective strategy. That is, they ask a smasher to play on a monitor and comment on how it feels. This strategy usually results in comments such as:
- “Oh it feels okay”
- “Mmm… I’m not sure, I think something might be slightly off.”
- “Yep, I’m pretty sure this thing lags.”
Fantastic responses! We have determined nothing. One major problem here is that smashers tend to placebo hard as soon as something that isn’t 2 feet in depth is placed in front of them. The other major problem is that humans are actually really bad at determining small fraction of a second differences.
With that said, it makes sense to attempt an objective strategy. With some help from Mofo, I developed a method for objectively testing lag on any type of screen. One of the beautiful things about this method is that it actually uses melee itself as part of the test. This means that we can say for a fact that every single possible source of lag has been accounted for.
As players, the lag we are sensitive to is the time between when an input is pressed to the time when the game’s reaction to that input appears on screen. In simpler words, lag is when stuff happens later than it should.
The game itself, however, cannot physically react to inputs immediately when they are received. This is because the game operates in discrete frames. For example, a character’s move can only ever begin on a frame, it cannot begin at any point in between. That said, players can and do press buttons in the time between frames. This means that if the A button is pushed to jab, the game will only begin the jab startup in a variable amount of time between 0 ms and 16.66 ms (length of one frame) – it will not start the jab immediately upon receiving the input.
Image 1: Comparison of different delays with identical frame output
The way this reality interacts with button combinations is rather interesting. It means that even if, on two separate occasions, the same two buttons are pressed with precisely the same delay in between, the same result is not guaranteed.
Image 2: Example of same inputs producing a different output
This means that even on a lagless monitor, there are two possible reasons why players are capable of being consistently technical.
- They understand the margin of error and press inputs in consequence to this. In the above graphic this would mean adding some time between button presses to guarantee the timing is always met
- The brain naturally synchronizes to the displayed refresh rate of the monitor.
The point to make here is that, even on a monitor that is lagless, lag still exists in some way. A fully lagless experience is impossible. One way or another, players are capable of dealing with some lag. Lag induced by a laggy screen, however, is an added constant on top of this variable lag – the effect of which will be explored later.
Synchronization by Sound
The original concept for determining the latency of a monitor was to somehow detect the time difference between when an input is pressed to when a particular frame shows up on screen. Unfortunately, this approach is iffy at best for the reasons described in the previous section. The result of such a test would be (true lag) + (time until next frame) where time until next frame is between 0 and 16.66 ms. Luckily, there is another signal that is separate from the video and yet is related to it time wise – sound.
Let’s consider Captain Falcon standing on FD about to falcon punch. When the b button is registered by the console, the console will send the video frame information consistent with displaying a falcon punch as well as the sound information containing the famous words “FALCON PUNCH!”. The sound and the video will always be sent out by the console at the same times respective to each other, irrespective of where the input landed in the subframe region.
When the video signal reaches the display, it is first processed, and then displayed. This processing time is effectively the lag of the monitor – it will cause the two signals, audio and video, to appear desynchronized. It is the amount of time of this desynchronization that will be measured to determine lag.
The Test Method
When a player pauses a match in melee, two events happen nearly simultaneously: a white decal surrounds the screen and a high pitched sound is played, both denoting the pause has happened. These are both very strong and easy to recognize signals.
The testing procedure is as follows:
- The audio output of the console is hooked up to an Arduino. The arduino lights an LED upon the detection of an audio signal. This effectively turns an audible signal (sound) into a visible signal (light).
- The video output of the console is plugged into the monitor.
- A high speed camera records the pause event.
- The high speed footage is analyzed and the time difference between when the LED lit up and the pause decal first became visible on screen is recorded.
When this test is executed on a CRT, the gold standard for response time, a time difference is obtained from step 4. This time difference is the expected value required if another system is to be called truly lagless.
When this test is executed on a laggy monitor, the time difference will be greater than what was seen on the CRT. The audio output will be detected at the same time but the pause decal will show up at some time later. The lag of this TV can then be calculated by the formula:
The gif below shows a CRT and a laggy LCD in a head to head comparison. The events were recorded individually and then combined. To best illustrate the comparison, the LED on time is set to 0 ms in both cases.
Image 3: Example test procedure – CRT vs Laggy TV
There are a few sources of error in the testing method. In order to improve the accuracy of the results, the test was executed multiple times on each monitor and the results averaged. That said, there is likely still about plus or minus 1 ms of error for the LCD results.
CRTs have less error associated with their measurement because determining when the video signal has appeared is less subjective. On an LCD, as can be seen in the gif above, the decal shows up in an incomplete fashion before the signal is accepted. This is done to provide a more fair comparison to the CRT – on which the top part of the decal is instantly fully clear.
Here are the averaged results for each monitor. (The BenQ RL2455HM monitor is the official MLG monitor)
Table 1: Comparison of lag on CRT and LCD TV’s
Graph 1: Graphical Comparison of lag via milliseconds (ms)
If you have ever been introduced to the website www.displaylag.com, you might wonder how it is possible for the results to be so low with the RL2455HM.
The RL2455HM monitor displays a frame from top to bottom. This method of showing a frame is identical to how a CRT displays a frame. Display lag database uses the average latency across three zones (top, center, and bottom). Using this metric, even a CRT would not have zero lag – it would have 8.3 ms of lag. This is because it takes a full 16.66 ms to display the entire frame from top to bottom. In this particular case and in many others, when comparing to a CRT it is more fair to subtract about 8 ms from the number reported by Display Lag.
Because CRTs actually take time to display a whole frame, it is technically possible for a flat screen monitor to appear faster than a CRT. Given a small initial delay such as the RL2455HM + LGP and a faster refresh rate, it may be possible for the center of a frame and certainly the bottom of a frame to appear on screen earlier than it otherwise would on a CRT.
Okay so using the RL2455HM + LGP setup induces about 2.86 ms of lag, but what does this actually mean to the player?
It is important to make a distinction between events that are affected by monitor lag and events that are not. Events that are executed via muscle memory timing, pressing one button at the correct timing after another, such as a wavedash are very easy to execute even on very laggy monitors because they do not utilize much visual prompting.
A good example of an event that would be affected by monitor lag is power shielding a laser, a 2 frame window.
Let’s consider a person who can successfully power shield a laser 95% of the time. Assuming that human reaction follows the gaussian pattern, a gaussian response that could meet this success rate has a mean at 1 frame before the laser hits and a variance of about 72.25 (ms). Introducing the lag of the monitor and assuming that the distribution is shifted over by an amount equal to the lag, the probability of a successful power shield only drops to 93.7%.
Image 4: Gaussian Distribution of Power Shielding with respect to Lag
Now a 95% success rate on a power shield is rather good. Let’s assume the person is still good but not super human – they have a lower success rate of 50% caused by an increase in variance. Given the same amount of lag, their success rate only drops to 49.7%. The takeaway from this is that given a higher variance, larger variation in a person’s ability to respond in a given amount of time, the effect of monitor latency diminishes.
It is also possible for the mean to not be perfectly centered along the target area. For example, consider a person has the same variance as described in the first example – a variance which signifies the person is quite proficient at hitting a 2 frame window. Now consider that this person tends to power shield late, late enough that their success rate is only 62.3% on a lagless system. With the lag added, this person’s success rate would drop to 49.1%. This scenario is just about the worst case given this kind of variance. The best case scenario is when the lag actually helps the player. If the same person had a tendency to hit early instead, their success rate would actually increase from 62.3% to 74.2%.
Notice that in the example where the percent of success dropped from 62.3% to 49.1%, the person was not extremely proficient at hitting the window to begin with. In contrast, when the success rate was 95% to start, the percent of success dropped a very small amount. A person proficient at hitting a 1 or 2 frame window either has a mean that is very close to centered on that target window, or has a very small variance. That said, there is a limit to how small a human’s variance can be. If a person has a 95%+ success rate hitting a 1 or 2 frame window, it is likely safe to assume it is caused by a well placed mean. Hence, players that can hit these timings very often will be very minimally affected by the added latency.
Now let’s talk about if the lag was a bit worse. Let’s consider a monitor that is slow by one full frame, 16.66 ms. In the first example with the 95% success rate person, their rate on this monitor would drop all the way to 50%. That is, 93.7% on a 2.86 ms monitor, 50% on a 16.66 ms monitor. This highlights the fact that there is a major difference between a monitor that is pretty good and one that is very good. Most monitors that people have tried would likely fall under the “pretty good” category at best. Do not allow past experiences with other monitors to influence your conception of these “very good” monitors.
All the calculations in this section were made under the assumption that the human does not adapt to the new lag. It may also be possible that the brain notices the slight offset and corrects to some degree. If the brain does do some correction, then the difference would be even smaller than described.
Screen Quality Perception
The topic of this section is factors other than lag which might cause a person to think negatively about a particularly monitor.
By the reasoning outlined in the previous section, minor lag does not appear to be a major factor for player performance. That said, I ask the reader, have you ever heard someone claim that some CRTs lag? Why do people think this? In my test results, CRT 2 is a 14 inch CRT. I have heard many negative comments about CRT 2. People just don’t seem to like it, often claiming that it lags. As shown by the results, the lag difference is essentially non-existent – it is well within the error of the test. So then, why do people not like it?
My theory is that people are also sensitive to image distortion. CRT 2 has a very clear image, but being a small monitor, it has a rather rounded screen. This rounded screen causes the image to appear somewhat distorted. This is very minor and difficult to notice but it may be the cause for the hate it has received.
Regarding CRTs, in my previous experience I tend to find that people like the larger, flatter screens.
The takeaway here is that lag is not the only problem with a screen. When an image looks different than another, it can throw a player off. This precise issue leads to one of the most powerful arguments for having a pro-LCD position. All the screens are the same. No more swapping back and forth between small CRTs and large CRTs. No more old, ugly, discolored CRTs. No more terrible terrible audio. The same image – same experience – every time.
It is true that very fast monitors such as the RL2455HM have some problems. Periods of fast movement can lead to minor ghosting. But overall the image quality is extremely good. After a few hours of using one, I fully expect a player to be used to it and be capable of ignoring any of its image defects.
We have seen that major tournament hosts and companies are reluctant to use CRTs. Maybe having an assortment of unique, archaic TVs gives their venue an unprofessional look. Maybe obtaining CRTs from the community is a hassle. Regardless of the reason, it is certainly a point which weakens these entities’ desire to host smash events.
That said, maybe they will accept our CRTs this year. But what about the next? And then the year after that? I expect if you are reading this you have a desire to see smash grow. CRTs are dead technology. Can we not adapt to changing technology? What kind of image does that portray to people that are not part of our scene?
There’s no game quite like melee. The fluidity of movement and execution skill cap enable a brilliant form of art we’ve come to worship. The love is real, the potential for growth is now.
Everyone has noticed the growth which our exposure at Evo provided. These big events are paramount to our continued growth. If dealing with an extremely small amount of delay helps aid that cause, how can you not support it? This small amount of lag, by the way, is bound to reduce even further as the technology improves. Maybe our exposure at these big events and our willingness to try new technology will encourage companies like BenQ or Asus to come out with new monitors that are even better for our use – monitors with support for native component input, or even maybe composite inputs.
To those that own these set ups or plan on getting one, I implore that you configure them properly and invite people to try them. For those that haven’t tried them, I encourage you to give it a fair chance. Who knows? You may find that these convenient monitors are not so bad after all.
Special thanks to:
Julien “Mofo” Heller – For helping me come up with the original concept for the test
Alex Jebailey – For suggesting the AverMedia LGP to the melee community and as such getting me interested in testing it
Tony “Aisight” Cheng – For doing the original monitor to monitor comparisons and being supportive of HD monitors
Sheridan and Tafokints – For the comments and feedback