log average miss rate may cause errors
Dear author, thank you for your contribution to this project. I found something wrong when calculating the log average miss rate parameter (function: bb. stat. lamr) using this project.
The log average miss rate is obtained through FPPI Miss Rate. Calculate by selecting an average of 9 points in the interval [10 ^ -2, 10 ^ 0].
When the curve ends prematurely, additional points are obtained through interpolation methods. When the value of curve y (i.e. Miss Rate) is small and does not change much, it is easy to interpolate to obtain 0. Afterwards, logging the values obtained by the interpolation method will result in errors (log 0 tends towards infinity).
Suggestion: Interpolating to obtain additional values is the most direct and correct method, but I haven't found a better solution for the problem of log0 yet. But I know from the link below that using the minimum value that the curve can take to replace missing values is also a feasible solution.
The above is my suggestion, I don't know if it will work.
Link address: https://zhuanlan.zhihu.com/p/665777082
chapter：fppi/fppw
（对于一些在达到特定FPPI值之前，就已经提前结束的曲线，miss rate值则取曲线所能达到的最小值）
def lamr(mr_fppi, samples=9):
"""Computes log average miss-rate from a MR-FPPI-curve.
The log average miss-rate is defined as the average of a number of evenly spaced log miss-rate samples
on the :math:`{log}(FPPI)` axis within the range :math:`[10^{-2}, 10^{0}]`
Args:
mr_fppi (pandas.DataFrame): Miss-rate and FPPI values
samples (int, optional): Number of samples within the given interval; Default **9**
Returns:
Number: log average miss-rate
"""
if len(mr_fppi) <= 1:
return float('nan')
m = np_col(mr_fppi, 'miss_rate')
f = np_col(mr_fppi, 'false_positives_per_image')
s = np.logspace(-2.0, 0.0, samples)
# This will lead to unsatisfactory results
# interpolated = interpolate.interp1d(f, m, fill_value=(1.0, 0.0), bounds_error=False)(s)
# Therefore, I made the modifications according to the method in the link
interpolated = interpolate.interp1d(f, m, fill_value=min(m), bounds_error=False)(s)
log_interpolated = np.log(interpolated)
avg = sum(log_interpolated) / len(log_interpolated)
return np.exp(avg)error message：
I have uploaded the results of the experiment (pandas type). This includes ground truth (annotations. xlsx) and the detection results of two models (model1.xlsx, model2.xlsx).
The log average miss rate of mode2 can be calculated to be around 0.074 using the Lamr function. However, due to the above reasons, the model can only obtain 0, and according to common sense, its result should be larger than that of model 1.
