How to use this tool?
First of all, select from the upper dropdown examples, or find out what kinds of things you want to compare.
It can be employee A and employee B, or it can be design A and design B,
or product A and product B, and so on. Anything.
You also need an event that can be numerically measured, that has several occurences. For example exam results,
incomes, satisfaction percentage indicies or something like these.
If you have these all, just fill in both input fields and you will get the averages, their rate compared to each other and the certainty of the result.
The names of cases can be rewritten, despite they are not input fields.
How does it work?
It's a two-sample two-tailed unpaired student t-test applied for differently distributed data, that tests null-hypothesis according to Welch algorithm, that there is no statistically significant difference.
But we don't want to bore you with math behind it, just use it. You don't need university-level math in order to use a tool that is based on that.
Why you can't be absolutely sure?
Because the difference between the averages of case A and case B may come from the work of coincidence.
Your data comes from an experiment or observation that is not exactly repeatable, they are not accurate, there is a fluke in them.
If you would measure again, you would get different values. This distribution causes that you can be sure
about that the average of one case is better than of the other, only if you have many numbers and the difference between averages is big.
The more data and the bigger difference in averages, the bigger the certainty.
Think about it: If there is a little difference in averages of exam results in a class between girls and boys, and the class size is small,
it may be just the effect of randomness, so you cannot say it with complete certainty that one gender performs better than the other regarding to the exam results.
What does the certainty percentage show?
You can easily calculate whether the average of case A or case B is better, but can you tell how sure you are?
Well, sometimes yes, sometimes no.
In case of few numbers and small difference in averages it is not sure that there is a real difference.
In case of many numbers and big difference in averages it is sure that there is a real difference.
But what if there are many numbers and small difference in averages, how sure are you? The above certainty value shows you as a percentage
how much you can be sure about the fact that the average of one case is better than the other.
Nevertheless it does not tell you anything about the strength of the difference.
If we are 100% sure that the average of A is better than B, it means only that A is better somewhat, but it cannot be told how much,
because you can conclude it only from the data of the experiment or observation.
What does low sureness mean?
It means that from these numbers it cannot be known whether the average of case A or case B is better. So it does not mean that
there is no difference and the difference experienced is the work of coincidence, but it means you cannot be sure, whether it is
the work of coincidence or a real difference.
The certainty is said to be low under 95%, so you can't be sure about the result. Generally, 95% or bigger sureness is required.
If you reach 99% or better, then you can be sure already. (But there is 1% chance, that the difference happened because of a very rare coincidence.)
What does high sureness mean?
It means that the average of one case is surely better than of the other.
Usually above 95% or 99% certainty level is considered to be high.
It's important that despite of the certainty being high, it only means that one case is better than the other,
but the difference between the averages of the two cases may be minimal.
This is why you must also check the experienced difference rate of averages.
How to increase the certainty?
You need more data. If you continue your experiment or observation with a larger number of events, you will get better certainty,
even if the averages or their difference rate don't change.