Hit Rate
Definition
The hit rate measures in a top-k evaluation for how many test cases a sought hit is in the result list.
Here, hit rate means the hit rate of a ranked list, not recall or cache metric. In the referenced recommendation experiment, one withheld target object is searched per test person. The hit ratio evaluates its occurrence among the top results and is averaged over test persons.
Simply explained
Imagine searching for a specific book per person in a short suggestion list. It is counted whether it is present, not its exact position. The comparison applies to this fixed test scenario.
Why it matters in practice
The metric describes success under a defined list length and test selection. These conditions must be stated for evaluation.
Example from business
Possible example: A team reserves a known suitable spare part for each test case and checks whether the recommendation list contains this part.
Limits and pitfalls
- Hit rate does not show how good all other suggestions are. A test with only one target object must not be equated without explanation with a recall evaluation over many relevant objects.
Origin & history
He et al. use Hit Ratio to evaluate recommendation lists in the cited 2017 version. Their protocol holds back one interaction per user and averages the results; this is not an original proof of the metric.
Use in mid-sized companies
- An internal recommendation test with documented target objects is conceivable. Candidate selection and list length should remain unchanged in comparisons.
Frequently asked questions
Does the hit rate consider the exact rank?
In the described hit test, only whether the target object appears within the list counts.
Is it equivalent to recall?
Not generally. Here, one hit per test case is evaluated, not the proportion of all relevant objects.
Sources
- Neural Collaborative Filtering · He et al. (2017): Neural Collaborative Filtering
Editorially reviewed on: