jeudi 12 décembre 2019

Benchmark a function over different inputs (of different sizes) in python efficiently

I'm trying to benchmark a function over different inputs which have different sizes.

For example I have a function something(array) that takes a numpy array with variable length as an input and a list of possible arguments args = [np.random.rand(3, 2, i) for i in range(1,1000)]. What I want to do now is to test the time something needs dependent on the input.

What I currently did is the following:

def iterator(ys, func):

    times = []
    for i in range(len(ys)):
        start = timer()
        jmp = func(ys[i])
        end = timer()
        times.append(end - start)
        if((i % 100) == 0):
            print(i/(len(ys) + 1)*100, '%')

    return times

With this I get the results I want. The problem is that with this method I do not get any key figures like the variance or mean for each input size. I could also do a second loop where I run the iterator several times and use the results to compute the key figures but I'd rather use an established method if there is one.

I tried to apply pytest-benchmark (https://pytest-benchmark.readthedocs.io/en/stable/) but was not able to find a way where I could pass a list of arguments to a function.

Maybe some of you have a nice solution to this!

Aucun commentaire:

Enregistrer un commentaire